| Time |
Nick |
Message |
| 07:42 |
|
collum joined #evergreen |
| 07:51 |
|
mantis1 joined #evergreen |
| 07:57 |
|
BDorsey joined #evergreen |
| 08:21 |
|
kworstell-isl joined #evergreen |
| 08:37 |
|
mmorgan joined #evergreen |
| 08:58 |
|
Dyrcona joined #evergreen |
| 09:05 |
|
dguarrac joined #evergreen |
| 09:15 |
Dyrcona |
miker: Well, ingest_ctl survived the other 157, 000 records yesterday with nothing much going on. I'm deleting a few more URIs and then going to load some as another test. |
| 09:16 |
* mmorgan |
added an additional commit to abneiman, jweston and jihpringle's release notes: https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/mmorgan/docs-releasenotes-2022-10-18 |
| 09:19 |
Dyrcona |
mmorgan++ |
| 09:20 |
miker |
Dyrcona: that's good news |
| 09:21 |
Dyrcona |
Yeah. I have no idea what happened at 17:20 on Monday. ingest_ctl just stopped running. |
| 09:22 |
Dyrcona |
I should have counted the records in this file of deletes. There are more than I thought. :) |
| 09:42 |
Dyrcona |
Hmm.. Eight of the new records failed. I wonder if I'm having encoding issues again. I swear I have fixed this, but every time I run these records from this vendor, it seems to be broken in new ways. |
| 09:43 |
Dyrcona |
Y'know. I should probably just set the program to do "the right thing" according to the Encode.pm documentation, and if that breaks this vendor's records, that's not my problem. --Heh. Yeah, right..... |
| 09:44 |
Dyrcona |
miker: Is there anyway to find error messages for the failed records? |
| 09:45 |
miker |
Dyrcona: depends on where the error occurred, and (assuming the answer to that is "inside postrgres") your logging level |
| 09:46 |
miker |
you can point the process_entry function directly at an entry by hand, in psql, if you want to, also |
| 09:47 |
|
jvwoolf joined #evergreen |
| 09:48 |
Dyrcona |
I don't see anything in the postgres logs other than the warning that ingest failed for queued entry #. |
| 09:48 |
Dyrcona |
OK. I'll stop ingest_ctl and do a couple by hand. |
| 09:50 |
Dyrcona |
Might be nice if we could capture postgres errors and log them, perhaps in a new column in the ingest_queue_entry table. |
| 09:57 |
miker |
seems doable, OTTOMH |
| 09:57 |
Dyrcona |
Yeah, I'll look into it. |
| 09:58 |
Dyrcona |
Also, I just processed one of the failed records and it didn't fail. |
| 09:58 |
Dyrcona |
I dumped the rows to a file, then set fail_time null, and ran the process function on the lowest id. |
| 09:58 |
Dyrcona |
I'll try the others. |
| 10:02 |
Dyrcona |
Grr. wanted to look one of these records up in the OPAC, and I get a 502 bad gateway.... |
| 10:02 |
Dyrcona |
Something's hosed in my apache/nginx configuration. |
| 10:05 |
Dyrcona |
Checking the database real quick looks like it actually worked for at least one of the failed records when I processed it manually. |
| 10:06 |
* Dyrcona |
keeps looking. |
| 10:12 |
Dyrcona |
My 502 is probably because I'm running an out of date Apache configuration. |
| 10:50 |
Dyrcona |
OK. Another batch of about 28,000 records to use for a stress test. |
| 10:55 |
|
stephengwills joined #evergreen |
| 10:55 |
Dyrcona |
Ugh. Forced to use screen because tmux is not installed.... |
| 10:56 |
stephengwills |
morning. if I RAISE some text in a postgresql function should I see that message in postgresql-10-main.log? or does it channel it somehwere else? |
| 10:59 |
stephengwills |
for context, I’m getting a fair number of duplicate key value violates unique constraint "browse_entry_sort_value_value_key" errors and am trying to view the search results in metabib.reingest_metabib_field_entries before it attempts to insert a sort_value that results in this unique constaint error. |
| 11:01 |
Dyrcona |
stephengwills: Depends, I think, on where and what you RAISE. I use RAISE NOTICE in DO blocks to get "output" for instance. It goes to the console. I don't think it ends up in the logs. |
| 11:01 |
Dyrcona |
Also, what are you doing when you get these duplicate key entries? |
| 11:05 |
stephengwills |
trying to process a vendor file with vandelay |
| 11:08 |
Dyrcona |
Is did you mean/symspell enabled? |
| 11:09 |
stephengwills |
I don’t know? this a library setting? |
| 11:10 |
Dyrcona |
What version of Evergreen? |
| 11:11 |
stephengwills |
3.7.1 |
| 11:12 |
Dyrcona |
You might have did you mean, but browse_entry duplicate keys can happen without it. The thing is, I've only ever seen it otherwise with pingest, which is why it does the browse all in one batch. |
| 11:13 |
Dyrcona |
I don't know how that happens with vandelay. |
| 11:17 |
stephengwills |
I’m seeing it enough so I was trying to determin why, if a key exists, it is attempting to insert into browse_entry at all? bringing me full circle to how to get the script talking… i.e. RAISE NOTICE arguments, etc. |
| 11:19 |
Dyrcona |
Yeah. I get that. I've only ever seen duplicate key errors from browse entry when parallel processes are working on related records, like 2 bibs from the same author. I didn't think Vandelay did parallel ingests. |
| 11:19 |
stephengwills |
I am watching one and it is working the batch sequentially |
| 11:19 |
Dyrcona |
Depending on your postgres log level, RAISEs should go into the logs. |
| 11:20 |
stephengwills |
oh, I should check the pg config. good idea. |
| 11:20 |
Dyrcona |
And, the pg documentation. |
| 11:20 |
stephengwills |
heh. I have a T-Shirt that warns against doing that. |
| 11:21 |
Dyrcona |
:) I think most of us have that shirt. |
| 12:02 |
|
jihpringle joined #evergreen |
| 12:34 |
|
collum joined #evergreen |
| 12:44 |
Dyrcona |
Try to rm a file, get told it doesn't exist. It's right there when I do ls. |
| 12:49 |
* Dyrcona |
suspects file system corruption. |
| 12:49 |
|
collum joined #evergreen |
| 13:06 |
csharp_ |
Dyrcona: aw fsck it! |
| 13:16 |
Dyrcona |
csharp_++ |
| 13:58 |
Dyrcona |
hm... ingest_ctl crashed again |
| 14:03 |
|
stephengwills_ joined #evergreen |
| 14:17 |
Dyrcona |
This time my console reports a problem with 8 of the records, but the log for the load only records a problem with 3 of those 8. |
| 14:17 |
Dyrcona |
I wonder if the bad records killed the ingest? |
| 14:18 |
Dyrcona |
Network/WiFi issues don't help. |
| 14:24 |
|
jihpringle joined #evergreen |
| 14:25 |
Dyrcona |
Nope. So far, running the function manually on these records works. |
| 14:29 |
Dyrcona |
First time that I've gotten 1 that is Overridden. |
| 14:35 |
Dyrcona |
Looks 3 with override_by set. One of those has been ingested. |
| 14:37 |
Dyrcona |
So, looks like the same record was updated more than once. |
| 14:40 |
Dyrcona |
I wonder if the 3 problem records have Windows smart quotes in them. IIRC, one of those looks like either the end of field or end of record character. |
| 14:44 |
mmorgan |
Smart quotes are kinda like smart TVs in that neither are all that smart :-/ |
| 14:46 |
Dyrcona |
mmorgan: True dat! |
| 14:47 |
Dyrcona |
The 3 records with override_by set do NOT coincide with "problem" records, jsut for the ... um ... record. |
| 14:49 |
Dyrcona |
And, whew! It's not a problem with my load program adding identical URIs, either. Both loads had a URI that matched the same record. |
| 14:50 |
Dyrcona |
So, a located URI for two libraries, one in each load. |
| 15:27 |
Dyrcona |
Five more failed. Guess I'll check them manually. |
| 15:29 |
Dyrcona |
I'll wait for the 1,000 or so pending inserts to finish. |
| 15:42 |
Dyrcona |
Yeahp. Five more failed records that process OK when done individually. |
| 15:48 |
Dyrcona |
Think I'll dump the marcxml from the failed records in case that will help later. |
| 16:17 |
Dyrcona |
abneiman++ |
| 16:21 |
|
jvwoolf left #evergreen |
| 16:32 |
|
sleary joined #evergreen |
| 16:40 |
|
kmlussier joined #evergreen |
| 16:41 |
kmlussier |
@quote add <mmorgan> Smart quotes are kinda like smart TVs in that neither are all that smart |
| 16:41 |
pinesol |
kmlussier: Error: You must be registered to use this command. If you are already registered, you must either identify (using the identify command) or add a hostmask matching your current hostmask (using the "hostmask add" command). |
| 16:41 |
kmlussier |
Hmmm...I thought I was registered. Now I have to relearn something. |
| 16:42 |
mmorgan |
:) |
| 16:42 |
Bmagic |
:) |
| 16:43 |
mmorgan |
pinesol, you know kmlussier!! |
| 16:43 |
pinesol |
mmorgan: No, you're a puzzleheaded kraken! |
| 16:43 |
kmlussier |
pinesol is angry at me for leaving, apparently. |
| 16:43 |
pinesol |
kmlussier: Beyond here be dragons. |
| 16:44 |
mmorgan |
pinesol needn't get personal :-/ |
| 16:44 |
pinesol |
mmorgan: I see nothing, I know nothing! |
| 16:45 |
kmlussier |
When I try to identify, it tells me I'm already identified. When did IRC become so difficult? |
| 16:46 |
kmlussier |
@quote add <mmorgan> Smart quotes are kinda like smart TVs in that neither are all that smart |
| 16:46 |
pinesol |
kmlussier: Error: You must be registered to use this command. If you are already registered, you must either identify (using the identify command) or add a hostmask matching your current hostmask (using the "hostmask add" command). |
| 16:46 |
* kmlussier |
shakes her first at pinesol |
| 16:47 |
Dyrcona |
@quote add <mmorgan> Smart quotes are kinda like smart TVs in that neither are all that smart |
| 16:47 |
pinesol |
Dyrcona: The operation succeeded. Quote #232 added. |
| 16:47 |
Dyrcona |
kmlussier: You must have known I was talking about you in the release channel! |
| 16:48 |
Dyrcona |
I opined how we sorely miss you and dbwells. |
| 16:48 |
kmlussier |
Dyrcona: Were you saying nice things? |
| 16:48 |
Dyrcona |
Yes. |
| 16:49 |
kmlussier |
Aw, thank you. I miss you all too, but I do try to help out behind the scenes sometimes. I made some wiki updates last weekend. |
| 16:49 |
Dyrcona |
Bmagic++ for adding https://wiki.evergreen-ils.org/doku.php?id=hack-a-way:hack-a-way-2022 |
| 16:49 |
Dyrcona |
kmlussier++ |
| 16:49 |
Dyrcona |
kmlussier: I heard you're doing a presentation on privacy at NELA. |
| 16:50 |
kmlussier |
Yes, Jeanette and I are sharing info on our respective privacy audits. I've already done something similar twice for a vendor group that will not be named in this channel. |
| 16:50 |
Dyrcona |
:) |
| 16:51 |
kmlussier |
Speaking of which, I just realized two days ago that the User Group meeting for this same vendor is the last week of April. Looks like I will only be able to join all of you for one day. |
| 16:51 |
Dyrcona |
Aw,shucks |
| 16:51 |
mmorgan |
:-( |
| 16:52 |
kmlussier |
Well, I probably will also drive up to Worcester after work. Bmagic: I'm assuming there will be games there? |
| 16:53 |
Dyrcona |
There better be games. I need to be a spy, again. |
| 16:54 |
kmlussier |
:) |
| 16:54 |
abneiman |
in the last outreach meeting we talked about "Evergreen Trivia" as conference social event and I feel like kmlussier would probably crush that |
| 16:55 |
kmlussier |
abneiman: I once considered the idea of Evergreen Trivia for an event at our vendor table one year. |
| 16:56 |
kmlussier |
I probably would only be good at who submitted various LP bugs. That knowledge still seems to sit in my head. |
| 16:56 |
abneiman |
ha! |
| 16:56 |
Dyrcona |
:) |
| 16:56 |
abneiman |
launchpad trivia: the game everyone loses??! :) |
| 16:57 |
Dyrcona |
I don't even remember bugs that I submitted, until I try to submit it again. |
| 16:57 |
kmlussier |
Here's a trivia question: Which Evergreen LP bug has the most comments? I bet the answer to that question hasn't changed in the last four years. |
| 16:57 |
abneiman |
Dyrcona, same |
| 16:59 |
Dyrcona |
That's a good question. I have no idea. I'd have to cheat. |
| 16:59 |
abneiman |
kmlussier, no idea here either! |
| 16:59 |
kmlussier |
I would say bug 1198465 |
| 16:59 |
pinesol |
Launchpad bug 1198465 in Evergreen "Support for Conditional Negative Balances" [Wishlist,Fix released] https://launchpad.net/bugs/1198465 |
| 16:59 |
mmorgan |
Oh yeah! THAT one! |
| 17:00 |
abneiman |
kinda disappointed it didn't break three digits, TBH |
| 17:00 |
* Dyrcona |
would rather forget..... |
| 17:00 |
abneiman |
kmlussier++ |
| 17:00 |
mmorgan |
kmlussier++ |
| 17:00 |
kmlussier |
Dyrcona: Sorry! My bp still goes up when I look at that one too. |
| 17:00 |
mmorgan |
oh and: |
| 17:00 |
mmorgan |
Dyrcona++ |
| 17:00 |
mmorgan |
Bmagic++ |
| 17:00 |
mmorgan |
abneiman++ |
| 17:01 |
* Dyrcona |
has clocked out and started on a rum and Coke, so nothing matters right now. :) |
| 17:01 |
Dyrcona |
mmorgan: *I* think the original implementation was superior to what ended up in Evergreen, but I'm biased. |
| 17:02 |
Bmagic |
lol |
| 17:02 |
Dyrcona |
I also think it's an issue of teminology getting in the way of understanding how something worked. |
| 17:02 |
* mmorgan |
is tempted to go back and look at the original implementation - but not that tempted ;-) |
| 17:02 |
kmlussier |
Dyrcona: Yeah, I think you need to add a couple more comments to that bug saying as much so that we can top 100. |
| 17:03 |
Dyrcona |
kmlussier++ |
| 17:03 |
kmlussier |
Anyway, thank you for helping me end my workday on a positive note #evergreen! Have a good night everyone! |
| 17:03 |
Dyrcona |
You, too1 |
| 17:03 |
Dyrcona |
! |
| 17:04 |
mmorgan |
Good night kmlussier!! |
| 17:04 |
kmlussier |
Enjoy that rum and coke. |
| 17:04 |
Dyrcona |
Thanks. I did! |
| 17:06 |
|
mmorgan left #evergreen |
| 17:06 |
* Dyrcona |
is going to be off tomorrow, so probably won't hang out in IRC. |
| 17:07 |
Dyrcona |
*poof* |
| 17:16 |
|
stephengwills_ left #evergreen |
| 21:01 |
pinesol |
News from commits: LP1981746 follow-up: docs and makefile updates <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=c84102daacd7ccb277c566d58cb5e387d4cf0f3e> |
| 21:01 |
pinesol |
News from commits: LP1981746: add Geo::Coder::Bing to geosort application module <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=79b5e124d1956af43e9482260d0bd2d1393fd136> |
