| Time |
Nick |
Message |
| 06:31 |
pinesol_green |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 07:11 |
|
agoben joined #evergreen |
| 07:26 |
|
rjackson_isl joined #evergreen |
| 08:02 |
|
rlefaive joined #evergreen |
| 08:30 |
|
ngf42 joined #evergreen |
| 08:42 |
|
mmorgan joined #evergreen |
| 08:47 |
|
Dyrcona joined #evergreen |
| 08:56 |
|
terran joined #evergreen |
| 09:19 |
|
yboston joined #evergreen |
| 09:22 |
|
kmlussier joined #evergreen |
| 09:24 |
Dyrcona |
So, the update of the visibility vector at the end of the 3.0.3 upgrade script blew up on my test database. |
| 09:24 |
Dyrcona |
Looks junk record(s). |
| 09:24 |
Dyrcona |
Looks like, that is... |
| 09:25 |
Dyrcona |
Fun part will be finding them. |
| 09:37 |
|
jvwoolf joined #evergreen |
| 09:41 |
Dyrcona |
Whee! Internal server errors. |
| 09:43 |
Dyrcona |
Call to [open-ils.circ.bre.holds.count] failed for session [1516891165.157756634.23931972033], thread trace [1]:\nCan't use an undefined value as an ARRAY reference at /usr/local/share/perl/5.22.1/OpenILS/Application/Circ/Holds.pm line 4266.\n\n\n\n : \n, referer: https://jasontest.cwmars.org/eg/opac/advanced?query=k-5;qtype=keyword;locg=1;detail_record_view=0;sort=poprel;pane=expert |
| 09:48 |
Dyrcona |
And, second try leads me to a syntax error in our custom templates: egweb: template error: file error - parse error - opac/parts/record/copy_table.tt2 line 111-112: unexpected token (%-)\n [% IF ctx.get_org_setting(CGI.param('loc') \n [%- IF has_parts == 'true' %], referer: https://jasontest.cwmars.org/eg/opac/advanced?query=k-5;qtype=keyword;locg=1;detail_record_view=0;sort=poprel;pane=expert |
| 09:48 |
Dyrcona |
Whee! |
| 09:48 |
* Dyrcona |
thinks he knows where this day is headed. |
| 09:48 |
mmorgan |
Bad technology day :-( |
| 09:50 |
|
derekz joined #evergreen |
| 09:50 |
* Dyrcona |
can't blame the technology for a syntax error. :) |
| 09:54 |
* mmorgan |
can think of ways to blame technology for that ;-) |
| 09:55 |
|
rlefaive joined #evergreen |
| 10:02 |
Dyrcona |
Looks like it was not just a syntax error, but some extraneous code. |
| 10:03 |
Dyrcona |
I deleted the [% IF ctx.get_org_setting(CGI.param('loc') |
| 10:08 |
Dyrcona |
And, found the record with a subfield code of " that causes the ingest to die. |
| 10:08 |
Dyrcona |
So, one thing lead to another. :) |
| 10:08 |
* Dyrcona |
can't English.... |
| 10:12 |
|
mmorgan joined #evergreen |
| 10:16 |
|
ngf42 joined #evergreen |
| 11:09 |
|
Christineb joined #evergreen |
| 11:11 |
|
mmorgan1 joined #evergreen |
| 12:20 |
|
jwoodard joined #evergreen |
| 12:33 |
|
JBoyer joined #evergreen |
| 12:42 |
|
khuckins joined #evergreen |
| 12:43 |
|
jihpringle joined #evergreen |
| 13:12 |
jeffdavis |
Is a sudden spike in the number of pcrud drones a cause for concern? |
| 13:17 |
jeffdavis |
Obviously drone exhaustion would be a problem. We're not at that point but we had about 20 pcrud child processes initiated all at once. |
| 13:26 |
Dyrcona |
jeffdavis: I wouldn't worry about it too much, but I might try looking for pcrud calls in the logs to see where they're coming from. |
| 13:49 |
pinesol_green |
[evergreen|Bill Erickson] LP#1739465 EDI consistent use of buyer org unit san - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=d05def6> |
| 13:51 |
jeffdavis |
Looks like part of the explanation for the pcrud drone spike was someone retrieving over 30 copies all at once via separate near-simultaneous calls to open-ils.actor.get_barcodes from somewhere in the web client. |
| 13:52 |
jeffdavis |
And the drone count was apparently already elevated from someone doing Retrieve Patron on a bunch of holds from the holds pull list. |
| 14:03 |
* Dyrcona |
wonders if uploading a file to item status would look like that? |
| 14:11 |
jeffdavis |
Dyrcona: yes I think that would do it, looks like it loops over barcodesFromFile and fetches each barcode individually (ultimately using get_barcodes to handle barcode completion) |
| 14:25 |
pinesol_green |
[evergreen|Bill Erickson] LP#1743608 Replace barcode discards NULL/accidental entries - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=982d999> |
| 14:28 |
kmlussier |
terran: We've noticed that the xul Novelist select issues - the ones related to XUL being too old to support a javascript function being used - have been resolved. Presumably through some action on the part of Novelist. |
| 14:28 |
kmlussier |
terran: I know your issues were different, and that they also involved Syndetics, but I was curious if you're Novelist started magically working too. |
| 14:29 |
terran |
kmlussier: Hmm... |
| 14:32 |
terran |
kmlussier: Novelist is still working in the opac but not in the web client nor xul for us (we're on 3.0.2) |
| 14:32 |
terran |
kmlussier: syndetics added content isn't working anywhere. We can see in the logs that it's finding the files on syndetic's servers, and it's showing us the section title but not the content |
| 14:32 |
|
rlefaive joined #evergreen |
| 14:32 |
kmlussier |
terran: :( |
| 14:34 |
terran |
I had Novelist turn off the |
| 14:34 |
terran |
"additional e-resources" section for us since it included some http links, but that didn't make any difference |
| 14:34 |
jeffdavis |
Dyrcona: thanks for that suggestion, I confirmed that uploading a file in web client Item Status causes a spike in pcrud drones. I'll report that to LP. |
| 14:35 |
Dyrcona |
jeffdavis++ |
| 14:36 |
Dyrcona |
terran: Are you seeing anything in the error console in the client? |
| 14:36 |
Dyrcona |
Or, did we cover that yesterday? |
| 14:36 |
terran |
We're not seeing any errors that seem like they might be problems with syndetics |
| 14:43 |
terran |
Novelist is giving a Type error in the opac, but it still loads. Haven't seen any novelist errors in the client, it's just not appearing. |
| 14:59 |
Dyrcona |
I was looking at something unrelated this morning in a 3.0.3 XUL client talking to a test VM. |
| 14:59 |
Dyrcona |
Novelist was working for me there, but I didn't pay any attention to it. |
| 15:00 |
terran |
Hmm - looks like js.tt2 is trying to call /js/dojo/dojo/openils_dojo.js that isn't there |
| 15:00 |
|
jihpringle joined #evergreen |
| 15:01 |
Dyrcona |
I'm not sure it has to be there. Let me check my vm. |
| 15:02 |
terran |
It may be an old call that can be removed then |
| 15:03 |
Dyrcona |
Could be. I don't have it on my vm and things are working. |
| 15:03 |
Dyrcona |
I'm not sure Novelist would be related to dojo. The Noveilist code is loaded from their servers. |
| 15:04 |
terran |
Yeah, I'm just looking at every error I notice at this point |
| 15:05 |
Dyrcona |
OK. |
| 15:06 |
terran |
I'm not seeing any errors that anything is trying to call anything inside openils_dojo.js, just that it's not loading |
| 15:08 |
|
khuckins_ joined #evergreen |
| 15:16 |
csharp |
jeffdavis: re: pcrud drones - we had to up our max_children significantly after moving to 3.0 |
| 15:17 |
csharp |
continuing to see load spikes, but I wanted at least a week's worth of "real" data without changing anything before looking at ways to mitigate them |
| 15:18 |
|
khuckins__ joined #evergreen |
| 15:28 |
dbs |
csharp: hmm, maybe we'll stay on 2.12 for a while, sounds like a single OpenSRF server instance might not be all that sustainable |
| 15:35 |
|
rlefaive joined #evergreen |
| 15:51 |
terran |
When you find out a staff person has been using the test server instead of production server for the past week... |
| 15:52 |
mmorgan |
Oops. |
| 15:54 |
JBoyer |
Thoroughly tested, though. |
| 15:54 |
terran |
Heh |
| 15:54 |
terran |
There is now a big red message on the splash page of the test server. |
| 15:54 |
rjackson_isl |
sounds like something I would do - but not for that extended a period! |
| 15:55 |
terran |
Waiting for the tickets to start coming in needing help finding out how many bills were paid, items were circulated, and accounts were created on that workstation. |
| 15:55 |
terran |
Ooh, I just remembered I have tomorrow off! YES! |
| 15:56 |
JBoyer |
Excellent timing on that one. |
| 15:56 |
rjackson_isl |
run terran run |
| 15:57 |
|
khuckins joined #evergreen |
| 16:04 |
jeffdavis |
csharp: We're actually seeing the elevated drone count on 2.12. Maybe I should raise max_children anyway though, we'll have to do it when we upgrade in a few months anyhow. :) |
| 16:14 |
* kmlussier |
hits sends on her email to terran before she leaves for the weekend. :) |
| 16:16 |
terran |
kmlussier: got it! |
| 16:29 |
dbs |
jeffdavis: we don't have too many people using the web staff client on 2.12 (mostly me); but maybe that explains some of the mysterious spikes we saw after our upgrade |
| 16:31 |
Dyrcona |
jeffdavis: Our experience has been the same as that of dbs on 2.12. We have a few people testing the web staff client, but it hasn't shown itself to be a real problem, yet. |
| 16:32 |
Dyrcona |
Actually, it seems a bit troubling that we may need to increase drone counts after the 3.0 upgrade. |
| 16:36 |
jeffdavis |
We have 3 or 4 libraries using the 2.12 web client and haven't noticed any problems in general so far (today's pcrud drone thing is unusual, and I think also partly XUL client related). |
| 16:37 |
jeffdavis |
Maybe our app servers are over-resourced :) |
| 16:39 |
kmlussier |
Sigh...Automatic publishing from the EG blog to the FB page appears to be broken again. :( |
| 16:48 |
|
jvwoolf left #evergreen |
| 17:04 |
|
mmorgan left #evergreen |
| 17:43 |
|
derekz left #evergreen |
| 17:46 |
jeffdavis |
bug 1745499 |
| 17:46 |
pinesol_green |
Launchpad bug 1745499 in Evergreen "Web client: Barcode file upload inflates pcrud drone count" [Undecided,New] https://launchpad.net/bugs/1745499 |
| 17:48 |
csharp |
jeffdavis++ |
| 17:51 |
csharp |
dbs: we're going to stick with the status quo through the weekend, then look at miker's suggestions for possible mitigation: http://irc.evergreen-ils.org/evergreen/2018-01-16#i_341694 |
| 17:51 |
csharp |
(some discussion of resources earlier that day too) |
| 17:52 |
csharp |
obviously cross-brick router stuff requires more than one server tho :-/ |
| 18:11 |
jeffdavis |
berick++ |
| 18:31 |
pinesol_green |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
