| Time |
Nick |
Message |
| 04:46 |
|
bshum joined #evergreen |
| 06:01 |
pinesol |
News from qatests: Failed Running perl live tests <http://testing.evergreen-ils.org/~live//archive/2022-05/2022-05-19_04:00:02/test.49.html> |
| 07:10 |
|
rjackson_isl_hom joined #evergreen |
| 07:33 |
csharp_ |
hmm - might be hitting bug 1931737 |
| 07:33 |
pinesol |
Launchpad bug 1931737 in Evergreen 3.8 "Did you mean breaks parallel reingest and causes deadlocks when loading/overlaying bib records in the client" [High,Confirmed] https://launchpad.net/bugs/1931737 |
| 07:52 |
|
collum joined #evergreen |
| 07:58 |
|
rfrasur joined #evergreen |
| 08:21 |
|
mantis joined #evergreen |
| 08:56 |
|
Dyrcona joined #evergreen |
| 09:01 |
csharp_ |
miker: FYI, I've applied your fixes for bug 1931737 to PINES production - FYI, I had to alter the XXXX.schema.dym_delayed_reify.sql script to add the creation of the search.symspell_dictionary_updates table |
| 09:01 |
pinesol |
Launchpad bug 1931737 in Evergreen 3.8 "Did you mean breaks parallel reingest and causes deadlocks when loading/overlaying bib records in the client" [High,Confirmed] https://launchpad.net/bugs/1931737 |
| 09:02 |
|
mmorgan joined #evergreen |
| 09:10 |
Dyrcona |
csharp_: I remember having to modify that db upgrade when I tested it on 3.7, but I forget exactly what I did. |
| 09:13 |
Dyrcona |
Yeah: https://bugs.launchpad.net/evergreen/+bug/1931737/comments/17 |
| 09:13 |
pinesol |
Launchpad bug 1931737 in Evergreen 3.8 "Did you mean breaks parallel reingest and causes deadlocks when loading/overlaying bib records in the client" [High,Confirmed] |
| 09:15 |
Dyrcona |
Oh, look! O/S updates! Systemd is updated. I'll probably have to reboot. |
| 09:15 |
csharp_ |
Dyrcona: thanks for the confirmation |
| 09:16 |
Dyrcona |
You could test the 3.7 branch if you're testing on 3.7. That might be useful. |
| 09:16 |
* Dyrcona |
should reboot the laptop. |
| 09:18 |
|
Dyrcona joined #evergreen |
| 09:32 |
csharp_ |
miker: upgrade script is also missing search.symspell_dictionary_reify |
| 09:42 |
* Dyrcona |
was going to look at the changes in the 3.7 branch, but I don't seem to have a copy on my laptop. I wonder where I pushed it from... |
| 09:43 |
Dyrcona |
oh, never mind. found it. :) |
| 09:47 |
Dyrcona |
csharp_: Are you commenting on the rebased branch? The things you have reported missing appear to be there in the original branch that I signed off on. |
| 09:50 |
csharp_ |
https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/lp1931737-dym-deadlock-plus-pingest was what I applied |
| 09:52 |
Dyrcona |
That's the new on. I haven't looked at it. |
| 09:52 |
Dyrcona |
I'll remove the sigenedoff tag for now. |
| 09:55 |
|
jvwoolf joined #evergreen |
| 10:27 |
* miker |
reads up... |
| 10:34 |
miker |
csharp_: I've only been pushing master/3.9 branches until it's accept{able|ed}, and the new commit with XXXX is atop the earlier one with WWWW (where symspell_dictionary_reify is defined), unless I misunderstand you. the newest commit is targeted at Dyrcona's pingest serialization (aka slow parallel ingest) concern, but IIUC Dyrcona, you've made 3.7 and 3.8 branches for the original commit? |
| 10:35 |
Dyrcona |
miker: Only 3.7. I think it worked at the time on 3.8/ |
| 10:36 |
miker |
IOW, first run WWWW, then XXXX. sorry if that wasn't obvious. I intentionally left them separable because the first commit had been signed off. I'm hesitant to push my own here (spread the knowledge around) but am willing to. |
| 10:37 |
miker |
Dyrcona: ah, thanks. I'd expect 3.8 to be fine but have 48903475 tabs open and didn't look again just now ;) |
| 10:48 |
Dyrcona |
I'll see if I can have another look later today or tomorrow. |
| 10:49 |
miker |
thanks |
| 10:51 |
|
Guest47 joined #evergreen |
| 10:53 |
miker |
and for the room, the reason 3.7 needed a separate upgrade script (that will necessarily be /overwritten/ by the 3.8+ version) is that 3.8 saw the introduction of case insensitive browse logic which changed the same core ingest function as the delayed DYM reification |
| 10:54 |
csharp_ |
miker: understood - I've now correctly applied WWWW and XXXX |
| 10:54 |
miker |
s/overwritten/superseded/ |
| 10:54 |
miker |
csharp_: cool, and no mention of SQL errors is a good sign :) |
| 10:55 |
csharp_ |
yes |
| 11:10 |
csharp_ |
also seeing fewer open-ils.actor spikes so far on PG 10 - could be speed improvements brought on by parallel queries? |
| 11:15 |
csharp_ |
...and just as I typed that... * open-ils.actor [61691] uptime=15:15:37 cputime=00:00:48 #drones=192/192 100% |
| 11:15 |
Dyrcona |
heh. That's always the way. :) |
| 11:15 |
csharp_ |
This is the Way |
| 11:16 |
Dyrcona |
I had a typo in a script I'm working on and was writing out updates all the way to the year 10,216 AD before I stopped it. :) |
| 11:25 |
* mmorgan |
waves as Dyrcona chases script back to the beginning of time :) |
| 11:25 |
Dyrcona |
More like 8,000 years into the future. :) |
| 11:25 |
mmorgan |
Oops, that would be the end of time. |
| 11:25 |
mmorgan |
:) |
| 11:25 |
Dyrcona |
Nah, we've got 300 million years or so before the Sun is too hot and Earth goes Venus. |
| 11:27 |
jvwoolf |
csharp_: Have you taken a look at the patch I've posted yet for bug 1968082? |
| 11:27 |
pinesol |
Launchpad bug 1968082 in Evergreen "Removing items from copy buckets leads to drone exhaustion" [Undecided,New] https://launchpad.net/bugs/1968082 |
| 11:29 |
jvwoolf |
Turns out item buckets was a big culprit for actor drones for us, I don't think we've had many instances of problems since applying this patch |
| 11:29 |
jvwoolf |
(And incidently, I need to figure out how to sign off on my commit after it's been pushed..._ |
| 11:30 |
Dyrcona |
jvwoolf: git commit --amend -s ; git push -f <remote> <branch> |
| 11:30 |
jvwoolf |
Dyrcona++ |
| 11:39 |
csharp_ |
jvwoolf: applied! |
| 11:39 |
csharp_ |
(just now) |
| 11:39 |
csharp_ |
jvwoolf++ |
| 11:47 |
jvwoolf |
So this command is failing. What am I missing? |
| 11:48 |
jvwoolf |
git push -f working user/jwoolford/lp1968082-batch-method-item-buckets |
| 11:48 |
Dyrcona |
Is that the name of the branch locally? |
| 11:49 |
jvwoolf |
The lp1968082-batch-method-item-buckets part is but not the full path |
| 11:49 |
Dyrcona |
If not, this should work: git push -f working HEAD:user/jwoolford/lp1968082-batch-method-item-buckets |
| 11:49 |
Dyrcona |
Assuming it is the currently checked out branch that you want to push. |
| 11:49 |
Dyrcona |
Ok, you can also replace HEAD with the local branch name if it's not checked out, or even if it is. |
| 11:50 |
Dyrcona |
I got into the habit of naming my local working branches the same as they're named on the server, so I don't have to worry about different names. |
| 11:50 |
jvwoolf |
Dyrcona++ # X 1000000 |
| 11:51 |
mmorgan |
Dyrcona++ |
| 11:51 |
mmorgan |
jvwoolf++ |
| 11:51 |
Dyrcona |
jvwoolf++ |
| 11:51 |
jvwoolf |
I should have asked last week when I was banging my head against this, but I think I was working on it Friday afternoon, which was a mistake! |
| 11:51 |
* Dyrcona |
thinks it is close enough for lunch |
| 12:03 |
|
jihpringle joined #evergreen |
| 12:11 |
Dyrcona |
Figured out my script's problem. I had a not in the wrong place. |
| 13:02 |
* mmorgan |
is frustrated by patron penalties and bug 1823225 and its friends. |
| 13:02 |
pinesol |
Launchpad bug 1823225 in Evergreen "Web client has no way to recalculate patron penalties" [High,Confirmed] https://launchpad.net/bugs/1823225 |
| 13:03 |
mmorgan |
I'm finding penalties out of sync with the actual status of the patron accounts, patrons not blocked when they should be, and worse, patrons blocked when they shouldn't be. |
| 13:04 |
mmorgan |
Anyone have a reliable way to sync up penalties with reality? |
| 13:13 |
Dyrcona |
mmorgan: On a patron by patron basis: open-ils.actor.user.penalties.update <auth> <usr.id> |
| 14:14 |
Dyrcona |
gmcharlt: Do you have any suggestions for pcrud or cstore calls that could be used to test bug 1974193? |
| 14:14 |
pinesol |
Launchpad bug 1974193 in OpenSRF "memory leak when C apps respond to client" [High,Confirmed] https://launchpad.net/bugs/1974193 - Assigned to Jason Stephenson (jstephenson) |
| 14:20 |
pinesol |
News from commits: LP#1968082: (follow-up) show progress bar when adding/removing items from bucket <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=d1bb45351ebc585a8f2ace174c63810b4e7a26b4> |
| 14:20 |
pinesol |
News from commits: LP#1968082: (follow-up) tidy up whitespace <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=5ddd6949c1112920296fb12c220eb84640a0fd2a> |
| 14:20 |
pinesol |
News from commits: LP#1968082: Use the batch API call to add and remove items from item buckets. <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=6f077bb931471c5a4018a6692805409f55d84ca7> |
| 14:27 |
gmcharlt |
Dyrcona: something like a few hundred calls to open-ils.pcrud open-ils.pcrud.search.bre "AUTHTHOKEN" {"id":{"=":A_BIB_ID}},{"flesh":1,"flesh_fields":{"bre":["call_numbers"]}} |
| 14:28 |
Dyrcona |
gmcharlt: Thanks. That will save me some time experimenting. |
| 14:28 |
gmcharlt |
small enough to not have responses chunked, big enough that it doesn't taken forever for see memory uasge growth |
| 14:45 |
gmcharlt |
Dyrcona: also, in case you didn't already see it, bug 1974193 is specifically relevant to pcrud memory leaks |
| 14:45 |
pinesol |
Launchpad bug 1974193 in OpenSRF "memory leak when C apps respond to client" [High,Confirmed] https://launchpad.net/bugs/1974193 - Assigned to Jason Stephenson (jstephenson) |
| 14:47 |
Dyrcona |
gmcharlt: Did you mean bug 1974195? I saw that one. Thought I'd code for one then the other. |
| 14:47 |
pinesol |
Launchpad bug 1974195 in Evergreen "memory leak when doing fleshed Fieldmapper searches from C DB services" [High,New] https://launchpad.net/bugs/1974195 |
| 14:48 |
gmcharlt |
whoops, yeah, that's the one I meant |
| 14:49 |
jeffdavis |
Would either of those memory leaks present as high memory usage by beam? |
| 14:49 |
Dyrcona |
I think I meant that I'd test one and then the other bug. Should I test the patches at the same time from the outset? |
| 14:50 |
|
jihpringle joined #evergreen |
| 14:51 |
gmcharlt |
no, they can be tested separately |
| 14:52 |
gmcharlt |
also, thanks for testing! |
| 14:52 |
Dyrcona |
Ok. I was thinking of testing the OpenSRF one and then adding the Evergreen one after without removing the OpenSRF patch. |
| 14:52 |
gmcharlt |
+1 |
| 14:53 |
Dyrcona |
jeffdavis: I don't think so. |
| 14:56 |
gmcharlt |
agreed, I wouldn't expect a direct connection |
| 14:57 |
Dyrcona |
I must be hungry. I typed './confiture ...' instead of ./configure. :) |
| 14:58 |
mmorgan |
./gofigure |
| 14:58 |
Dyrcona |
mmorgan++ |
| 15:25 |
Dyrcona |
Maybe I should have scripted this. :) |
| 15:51 |
jeffdavis |
another new open-ils.actor parallel requests fix: bug 1946531 |
| 15:51 |
pinesol |
Launchpad bug 1946531 in Evergreen 3.8 "Use batch method when adding users to bucket" [Undecided,New] https://launchpad.net/bugs/1946531 |
| 15:53 |
jvwoolf |
jeffdavis++ |
| 15:53 |
jvwoolf |
Today is a good day for performance bug fixes, it seems |
| 15:54 |
jeffdavis |
Gotta catch 'em all! |
| 16:09 |
Dyrcona |
gmcharlt++ The OpenSRF patch appears to work. The pcrud drone grows a smidge, then stays there. |
| 16:10 |
gmcharlt |
cool |
| 16:13 |
Dyrcona |
It even seems to help with the Evergreen pcrud bug as the drone seems grow more slowly. First time it rocketed right up to 128KB. After 3 or 4 iterations of the pcrud search for id >= -1, it's still below 100KB. |
| 16:23 |
Dyrcona |
It looks like there's a small memory leak somewhere else, still. With the Evergreen patch applied the drone still grows by 204 bytes per search request, but that's a lot better than what it was doing. |
| 16:25 |
Bmagic |
Dyrcona++ |
| 16:29 |
Dyrcona |
I'll test this more tomorrow and make a note of the numbers that I'm seeing for the bug. The patches definitely help! |
| 16:45 |
* miker |
wonders which struct ends up exactly 204B long and exists once for each session... (but I'll take 200B leak over a "size of every message that's not chunked" leak :) ) |
| 16:46 |
Dyrcona |
miker: It seems to vary a bit from 180 or so to almost 240 bytes. I had 2 or 3 in a row that were 204 bytes. |
| 16:46 |
Dyrcona |
I only noticed those and did the math when I said 204 bytes. |
| 16:47 |
miker |
ah! different api name? or always the same one but with different leak sizes? |
| 16:48 |
Dyrcona |
I was hitting up arrow in srfsh, so always the same request, the one from https://bugs.launchpad.net/evergreen/+bug/1974195/comments/2 |
| 16:48 |
pinesol |
Launchpad bug 1974195 in Evergreen 3.8 "memory leak when doing fleshed Fieldmapper searches from C DB services" [High,Confirmed] |
| 16:49 |
Dyrcona |
Without the patch, the drone grew very quickly to double or triple in size after a few requests, so definitely much better. |
| 16:52 |
Dyrcona |
I'll do this all again tomorrow on different vms and record the sizes of the drones both before and after the patches are applied. |
| 16:52 |
* Dyrcona |
calls it a day. |
| 17:03 |
|
mmorgan left #evergreen |
| 17:13 |
|
pete joined #evergreen |
| 17:19 |
|
jvwoolf left #evergreen |
| 17:23 |
csharp_ |
jeffdavis++ |
| 18:01 |
pinesol |
News from qatests: Failed Running perl live tests <http://testing.evergreen-ils.org/~live//archive/2022-05/2022-05-19_16:00:04/test.49.html> |
| 18:21 |
pinesol |
News from commits: lp1950507 Providers Admin Screenreader form fix <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=df9000506e75d3331a993c6e27ba6e0278b789ee> |
| 19:59 |
|
jihpringle joined #evergreen |
| 20:21 |
pinesol |
News from commits: LP1857910 release note blurb for field doc port <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=0ebada37a652f3e2b316714794391e2a849d5cb6> |
| 20:21 |
pinesol |
News from commits: LP1857910 add org family selector to field doc UI <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=d0713fac29e96dc7eb5643a13ad796b5bba92a1d> |
| 20:21 |
pinesol |
News from commits: LP1857910 make field doc grid sortable/pageable <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=1fc37ba2715922da2a076b2c7e968fb924a2ff61> |
| 20:21 |
pinesol |
News from commits: LP1857910 ws setting type for field doc grid <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=581c25ffa69ef776702a1c166303670fce65e842> |
| 20:21 |
pinesol |
News from commits: LP1857910 Field doc UI Angular 10 updates <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=6559f7d9d2896a225bc9224cdb0f362eb0bfffe7> |
| 20:21 |
pinesol |
News from commits: lp1857910 Field Documentation Port <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=7395d1ce56f1a780809af1db2b1df4a92919247d> |
| 20:21 |
pinesol |
News from commits: LP#1946531: (follow-up) show progress bar when adding users to bucket <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=80347b51c59cb9bda3bbaa85ee7919a4df626f64> |
| 20:21 |
pinesol |
News from commits: LP#1946531: use batch method to add users to bucket via User Buckets UI <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=252f8e5678508a81428067b1a5fac20baf4c048e> |
| 20:33 |
|
jihpringle joined #evergreen |
