| Time |
Nick |
Message |
| 02:07 |
|
phasefx joined #evergreen |
| 02:07 |
|
abneiman joined #evergreen |
| 02:08 |
|
degraafk joined #evergreen |
| 02:12 |
|
berick joined #evergreen |
| 06:01 |
pinesol |
News from qatests: Failed Running perl live tests <http://testing.evergreen-ils.org/~live//archive/2022-05/2022-05-23_04:00:02/test.49.html> |
| 07:28 |
|
rjackson_isl_hom joined #evergreen |
| 07:51 |
|
RFrasur joined #evergreen |
| 07:58 |
|
mantis1 joined #evergreen |
| 08:40 |
|
mmorgan joined #evergreen |
| 09:10 |
|
mantis2 joined #evergreen |
| 09:12 |
|
jvwoolf joined #evergreen |
| 09:15 |
|
collum joined #evergreen |
| 09:22 |
|
mantis1 joined #evergreen |
| 09:34 |
|
Dyrcona joined #evergreen |
| 10:23 |
Dyrcona |
Whee! Have five pcrud drones each using over 6% of RAM. |
| 11:52 |
|
mantis2 joined #evergreen |
| 12:04 |
pinesol |
News from commits: fix typo in 3.8.1 release notes <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=874f426f8f37fea1c09b51b114f1eb8709f6dbf6> |
| 12:28 |
|
rjackson_isl_hom joined #evergreen |
| 12:32 |
csharp_ |
looks like actor.usr_delete (specifically purge data) is causing the same sort of problems as bug 1931737 |
| 12:32 |
pinesol |
Launchpad bug 1931737 in Evergreen 3.8 "Did you mean breaks parallel reingest and causes deadlocks when loading/overlaying bib records in the client" [High,Confirmed] https://launchpad.net/bugs/1931737 |
| 12:33 |
csharp_ |
this comes from a user merge where the "old" user is a cataloger with lots of bib edits/changes |
| 12:38 |
mmorgan |
csharp_: Interesting. Years ago we had problems deleting staff users, so we have since just made them inactive instead. |
| 12:45 |
csharp_ |
mmorgan: your comment made me realize this issue is probably unrelated to DYM and has always been a problem |
| 12:46 |
Dyrcona |
csharp_: No, I don't think it is at all the same issue. Deleting staff and patrons with lots of transactions or items takes a long time. It can time out in the client. |
| 12:46 |
Dyrcona |
It's particularly bad for acquisitons catalogers. |
| 12:47 |
Dyrcona |
We delete them in the database when necessary. |
| 12:47 |
Dyrcona |
Merges can fail for the same reason. |
| 12:47 |
csharp_ |
yeah, it's the same logic |
| 12:48 |
csharp_ |
okay - thanks for helping me find the right track |
| 12:48 |
csharp_ |
mmorgan++ Dyrcona++ |
| 12:49 |
Dyrcona |
I don't think there are any deadlocks. It just times out. |
| 12:49 |
csharp_ |
in my case it wasn't deadlocks, but the purge was locking the bib table too long and queries were piling up |
| 12:51 |
Dyrcona |
OK. That is technically deadlocking, then. I stand corrected. |
| 12:52 |
Dyrcona |
We do scheduled SQL updates for library data on Tuesday and Thursday nights. The deletes of problems users are done during those updates. |
| 12:52 |
berick |
csharp_: curious if your internal flag for ingest.reingest.force_on_same_marc is enabled. |
| 12:52 |
Dyrcona |
So, I recommend doing the delete in the evening/night when the system is less busy. |
| 12:54 |
csharp_ |
berick: it is disabled - I did think to check that because it doesn't make sense to me that MARC updates would be necessary |
| 12:56 |
* berick |
nods |
| 12:56 |
csharp_ |
looks like biblio.extract_fingerprint(text) is running when I Ctrl-C |
| 12:57 |
csharp_ |
pretty consistently |
| 12:57 |
csharp_ |
I may RAISE more NOTICEs to see if I can glean more clues |
| 13:05 |
csharp_ |
wondering if we shouldn't also not do the fingerprint stuff if the MARC hasn't changed |
| 13:06 |
mmorgan |
Certainly no need to redo the fingerprint. |
| 13:08 |
Dyrcona |
I concur. |
| 13:10 |
csharp_ |
is there any reason why fingerprinting shouldn't just be part of biblio.indexing_ingest_or_delete |
| 13:10 |
csharp_ |
? |
| 13:11 |
csharp_ |
I've not done much with the guts of this part of the code, so I'm learning a lot on the fly |
| 13:16 |
Dyrcona |
I suspect that's just an artifact of development. |
| 13:17 |
|
mantis1 joined #evergreen |
| 13:50 |
|
mantis1 joined #evergreen |
| 13:55 |
|
jvwoolf joined #evergreen |
| 14:31 |
JBoyer |
I think the fingerprints may be related to metarecords also, so you can gather all members of an MR group by matching fingerprints rather then running a function on every record. |
| 14:32 |
miker |
csharp_: the main reason is conceptually the fingerprint layout is for MR mapping and unrelated to search indexing per se |
| 14:32 |
JBoyer |
That definitely doesn't mean it should be run unless the marc changes (or the fingerprinter does, which would then require manually rebuilding them directly) |
| 14:41 |
csharp_ |
JBoyer: miker: my strategy so far is to check the 'ingest.reingest.force_on_same_marc' within the trigger def - not sure that's the correct place, but just experimenting |
| 14:41 |
csharp_ |
within the fingerprint trigger def, that is |
| 14:42 |
JBoyer |
That should cover it. Even if the fingerprinting does change (again) the rebuild would be a one-off thing. |
| 14:46 |
csharp_ |
also looking at disabling those flags when doing the usr data purge |
| 14:50 |
Dyrcona |
I'd be inclined to make the fingerprint function into a regular function, to call the fingerprint function from the ingest function, and have the fingerprint function be called manually if needed at other times. |
| 14:50 |
csharp_ |
my main takeaway is there's room for some optimization that would help my specific issue and possibly all bib imports |
| 14:54 |
Dyrcona |
Yes, it looks like it would help everyone in the long run. |
| 15:37 |
csharp_ |
argh - been barking up the wrong tree for hours... |
| 15:37 |
csharp_ |
silly rabbit, you can't change internal flags on the fly from within a function |
| 15:38 |
csharp_ |
at least it's not working for some reason |
| 15:53 |
Dyrcona |
Depends on how and where the flags are used. Changing them in a transaction should work for that transaction. |
| 17:04 |
|
mmorgan left #evergreen |
| 17:07 |
|
jvwoolf left #evergreen |
| 18:01 |
pinesol |
News from qatests: Failed Running perl live tests <http://testing.evergreen-ils.org/~live//archive/2022-05/2022-05-23_16:00:02/test.49.html> |
| 22:35 |
|
DaMobi joined #evergreen |
