| Time |
Nick |
Message |
| 01:21 |
|
bshum joined #evergreen |
| 03:19 |
|
yar joined #evergreen |
| 06:10 |
|
jeff joined #evergreen |
| 06:17 |
|
Guest94274 joined #evergreen |
| 06:31 |
pinesol_green |
News from qatests: Test Failure <http://testing.evergreen-ils.org/~live> |
| 07:05 |
|
rjackson_isl joined #evergreen |
| 07:06 |
|
rjackson_isl_ joined #evergreen |
| 08:05 |
|
rlefaive_ joined #evergreen |
| 08:06 |
|
jvwoolf joined #evergreen |
| 08:12 |
|
rlefaive joined #evergreen |
| 08:14 |
|
jvwoolf1 joined #evergreen |
| 08:15 |
|
rlefaive joined #evergreen |
| 08:26 |
|
rlefaive joined #evergreen |
| 08:38 |
|
rlefaive joined #evergreen |
| 08:44 |
|
rlefaive joined #evergreen |
| 08:47 |
|
mmorgan joined #evergreen |
| 08:48 |
|
rlefaive joined #evergreen |
| 08:57 |
|
kmlussier joined #evergreen |
| 08:59 |
|
bos20k joined #evergreen |
| 09:01 |
|
rlefaive joined #evergreen |
| 09:06 |
|
rlefaive joined #evergreen |
| 09:06 |
kmlussier |
Good morning #evergreen! |
| 09:07 |
|
terran joined #evergreen |
| 09:07 |
|
Dyrcona joined #evergreen |
| 09:10 |
mmorgan |
kmlussier: Good Morning! |
| 09:11 |
|
rlefaive joined #evergreen |
| 09:21 |
|
rlefaive joined #evergreen |
| 09:23 |
|
rlefaive joined #evergreen |
| 09:26 |
csharp |
Good morning to all of y'all |
| 09:36 |
kmlussier |
csharp: Good morning! How did things go with the upgrade? |
| 09:39 |
|
yboston joined #evergreen |
| 09:49 |
dbs |
Morning! |
| 09:52 |
|
jvwoolf joined #evergreen |
| 10:03 |
|
rlefaive joined #evergreen |
| 10:04 |
Dyrcona |
Anyone else seeing not connected to the network errors that appear to be related to search.facets_for_record_set |
| 10:05 |
terran |
We're still getting reports of Hatch not working on 32-bit versions of Windows 7. Anyone else? |
| 10:07 |
csharp |
kmlussier: so far so good - the next couple of ours will be our true test |
| 10:07 |
csharp |
s/ours/hours/g |
| 10:08 |
csharp |
Dyrcona: we're not at this point |
| 10:09 |
Dyrcona |
csharp: OK. We get them infrequently, now. |
| 10:09 |
csharp |
we're seeing something that attempts to insert a new actor.card object with a null barcode |
| 10:09 |
Dyrcona |
And, it always seems to be related to a JSON query with that function. |
| 10:10 |
csharp |
2018-01-16 10:09:23 brick02-head open-ils.cstore: [ERR :5860:oils_sql.c:2522:1516115171317224] open-ils.cstore ERROR inserting actor::card object using query [INSERT INTO actor.card (active,barcode,id,usr) VALUES ('f',DEFAULT,DEFAULT,759589);]: 3505682 3505682: ERROR: null value in column "barcode" violates not-null constraint#012DETAIL: Failing row contains (5971016, 759589, null, f). |
| 10:10 |
Dyrcona |
Hm... NULL barcode could be staff, maybe. |
| 10:11 |
mmorgan |
Dyrcona: We have not seen any to speak of since lp 1704396 was resolved |
| 10:11 |
pinesol_green |
Launchpad bug 1704396 in Evergreen 2.12 "Slowness for metarecord and one-hit searches in 2.12" [High,Fix released] https://launchpad.net/bugs/1704396 |
| 10:11 |
csharp |
well, it appears to happen on a patron update |
| 10:11 |
|
rlefaive joined #evergreen |
| 10:12 |
Dyrcona |
mmorgan: Thanks. We have that fix applied and this is the only thing I've since the fix. |
| 10:12 |
Dyrcona |
It happens maybe once or twice a week. |
| 10:12 |
Dyrcona |
csharp: I haven't had any reports of that one. |
| 10:13 |
Dyrcona |
I could check my logs though. |
| 10:19 |
Bmagic |
performing a bib load of 35k bibs. I am getting ERROR: index row size 2952 exceeds maximum 2712 for index "browse_entry_sort_value_value_key" |
| 10:21 |
|
rlefaive joined #evergreen |
| 10:21 |
Bmagic |
Does that mean I have to do something with MD5? Seems a bit excessive. https://github.com/doorkeeper-gem/doorkeeper/wiki/How-to-fix-PostgreSQL-error-on-index-row-size |
| 10:22 |
dbs |
Bmagic: I feel like we talked about this before... approach would be to follow the metabib.full_rec view over metabib.real_full_rec with "substring"(real_full_rec.value, 1, 1024) AS value as the index on the view |
| 10:24 |
Bmagic |
dbs: I apologize. I do not recall having to change the db for a bib load. |
| 10:24 |
Bmagic |
dbs++ |
| 10:26 |
Bmagic |
looking at the state of the view I find - CREATE OR REPLACE VIEW metabib.full_rec AS ....... "substring"(real_full_rec.value, 1, 1024) AS value, real_full_rec.index_vector FROM metabib.real_full_rec; |
| 10:27 |
* csharp |
finally sees bug 1738488 emerge |
| 10:27 |
pinesol_green |
Launchpad bug 1738488 in Evergreen 3.0 "Web client: patron billing history results in long running query" [Medium,Confirmed] https://launchpad.net/bugs/1738488 |
| 10:27 |
Bmagic |
csharp: getting some of those crop up? |
| 10:27 |
csharp |
in our case it runs for about 4 minutes and so far not causing pile-ups |
| 10:28 |
Bmagic |
csharp: they require that the staff browse to that section of the patron account |
| 10:28 |
Dyrcona |
csharp: Haven't seen that. Just the payments one. |
| 10:28 |
Dyrcona |
We're not using 3.0 in production, but we do have it on a test server with production data. |
| 10:28 |
Bmagic |
csharp: and it doesn't happen for ALL patrons. Just some. Probably those with more than X number of historic bills..... |
| 10:31 |
csharp |
yeah makes sense |
| 10:32 |
jeff |
reminds me of our 0.00 bills issue. :-) |
| 10:32 |
Bmagic |
csharp: Take a look at that explain analyze, rows=44575740147225592344870912 |
| 10:33 |
csharp |
hmm, so what does open-ils.fielder do? We appear to be exhausting the max_children limit for it |
| 10:35 |
dbwells |
Bmagic: dbs: sounds like bug #1695911 |
| 10:35 |
pinesol_green |
Launchpad bug 1695911 in Evergreen "Long browse entries cause index row size error" [Undecided,New] https://launchpad.net/bugs/1695911 |
| 10:37 |
Bmagic |
dbwells: yep, that's the one |
| 10:38 |
dbs |
dbwells++ |
| 10:38 |
Bmagic |
right now, I am loading them 500 at a time to find the offender |
| 10:38 |
* dbs |
is not sure how or if he ever resolved that issue |
| 10:39 |
dbs |
I mean, on the one hand the data is pretty weird; on the other, many academic libraries would have ECCO |
| 10:39 |
miker |
csharp: fielder backs many grids in the web staff client, but not a lot in the xul client. you def want to increase from the default |
| 10:40 |
dbwells |
Yes, I think all of our cases also come from ECCO. |
| 10:40 |
|
rlefaive joined #evergreen |
| 10:40 |
|
remingtron joined #evergreen |
| 10:41 |
Bmagic |
csharp: just curious, what value do you have for that? |
| 10:41 |
csharp |
miker: thanks - will do |
| 10:41 |
csharp |
Bmagic: 50 right now |
| 10:41 |
csharp |
we'll need to up it - not sure what a sane value is |
| 10:42 |
dbwells |
And maybe some from EEBO. |
| 10:42 |
Bmagic |
our min is 5 with max of 15, max requests 1000 |
| 10:53 |
csharp |
Bmagic: miker: our max_requests for open-ils.fielder is at 17 - would raising that be better than upping the number of children? |
| 10:53 |
csharp |
(currently max_children is at 50) |
| 10:53 |
|
rlefaive joined #evergreen |
| 10:53 |
berick |
csharp: which max_requests? |
| 10:53 |
berick |
perl has 2 |
| 10:54 |
berick |
inside and outside of unix_config |
| 10:54 |
csharp |
oh |
| 10:54 |
berick |
the one inside is the important one |
| 10:54 |
csharp |
oh I see - inside is at 1000 |
| 10:54 |
csharp |
so I guess upping the max_children makes sense |
| 10:54 |
csharp |
gonna edge it up to 72 |
| 11:04 |
|
tlittle joined #evergreen |
| 11:05 |
|
jwoodard joined #evergreen |
| 11:07 |
|
Christineb joined #evergreen |
| 11:07 |
csharp |
and now open-ils.actor apparently needs adustment |
| 11:07 |
csharp |
also normal? |
| 11:10 |
|
rlefaive joined #evergreen |
| 11:12 |
Dyrcona |
Not sure, but we're not on 3.0, yet. |
| 11:12 |
csharp |
well, open-ils actor fell offline - gonna look into that, but on one or two of our six heads, we're seeing high percentage of open-ils.cat drones in use |
| 11:12 |
Dyrcona |
I thought I made a note of what to look for in the logs when you run out of children, but I can't find it. |
| 11:13 |
csharp |
server: no children available, waiting... consider increasing max_children for this application higher than 15 in the OpenSRF configuration if this message occurs frequently |
| 11:14 |
Dyrcona |
csharp: Thanks. I thought there was another string, but no children available works. |
| 11:15 |
Dyrcona |
Oh, and back to the null barcode. I'm seeing some of those messages this month, but no one has opened a ticket about problems. |
| 11:16 |
Dyrcona |
Looks like I get multiples for the same patron in a short period of time. |
| 11:16 |
* Dyrcona |
forgot he did that search. :) |
| 11:17 |
berick |
csharp: one thing that may be at issue here.. when using websockets there is no limit to the number of parallel requests hitting the server from a single client. (in XUL xhttpreq i believe 8 is the max). it's possible requests are coming in bigger bursts now. |
| 11:17 |
csharp |
looks like we have a helpdesk that points in that direction - they got a "database update failed" error when replacing a barcode |
| 11:18 |
csharp |
berick: makes sense |
| 11:18 |
csharp |
berick: it does appear to be hitting a single app server at a time - one is overloaded while the others are fine |
| 11:19 |
berick |
csharp: as in, it moves around? |
| 11:19 |
csharp |
right |
| 11:19 |
Dyrcona |
I'm looking for 'no children available' now, but I hope to not find any. :) |
| 11:20 |
Dyrcona |
Well, that would make sense if websockets is talking to the local brick. |
| 11:22 |
|
jihpringle joined #evergreen |
| 11:23 |
berick |
csharp: a useful bit of data when that happens would be to see if the bursts are coming from just a few clients (or a single client). grepping activity log for osrf_websocket_translator and open-ils.actor and grouping on the IP address or authtoken |
| 11:24 |
berick |
if we see one client sending like 50 requests at once, we need to fix that client UI and probably add rate limiting to the websocket translator |
| 11:28 |
|
rlefaive joined #evergreen |
| 11:44 |
|
ngf42 joined #evergreen |
| 11:45 |
|
rlefaive joined #evergreen |
| 11:46 |
csharp |
berick: thanks - I'll look into ggetting that |
| 11:47 |
csharp |
also, looks like the thing that broke open-ils.actor was someone pasting in what looks like the text of a formatted document into a barcode input field |
| 11:47 |
Dyrcona |
Oh, gotta love that. |
| 11:48 |
Dyrcona |
Bad paste. |
| 11:48 |
csharp |
yep |
| 11:49 |
csharp |
and I have nailed down a workflow that causes the null barcode thing - when replacing a barcode, each time you click "Replace Barcode" in the patron edit UI it adds a row with no barcode - then when saving, it tries to create the new one (only visible when clicking "See All") and that triggers the error |
| 11:51 |
csharp |
also, while See All *shows* the barcode rows, you can't delete them so the only thing to do is reload the patron and start over |
| 11:52 |
Dyrcona |
csharp++ |
| 11:52 |
Dyrcona |
I'd say Lp that, particularly if it is reproducible in the web staff client. |
| 11:54 |
mmorgan |
csharp++ |
| 11:55 |
* csharp |
already has the bug screen open |
| 11:59 |
miker |
csharp: cross-brick router registration will save you from those per-brick spikes. it spreads the requests around, round-robin style |
| 12:02 |
csharp |
miker: any examples of that around? I've not come across that before... |
| 12:05 |
* csharp |
reports https://bugs.launchpad.net/evergreen/+bug/1743608 |
| 12:05 |
pinesol_green |
Launchpad bug 1743608 in Evergreen "Web client: Too easy to unwittingly create null patron barcodes" [Undecided,New] |
| 12:07 |
dbs |
csharp++ |
| 12:09 |
csharp |
miker: do you mean ejabberd-level registration of the "router" user |
| 12:09 |
miker |
csharp: it's just where you have 1) non-localhost IPs and network-unique DNS (or /etc/hosts) names for your xmpp listeners rather than {private|public}.localhost 2) list multiple routers in your opensrf_core.xml, in the //config/opensrf/routers section |
| 12:09 |
csharp |
? |
| 12:10 |
csharp |
ok - we have 1) - I'll look into 2) |
| 12:12 |
|
khuckins joined #evergreen |
| 12:14 |
csharp |
miker: so <name> would become "router-brick01", "router-brick02" etc. or would they all be named "router"? |
| 12:15 |
miker |
csharp: all just router, unless you have different xmpp users registered for the router on each brick |
| 12:15 |
csharp |
oh, it's per domain |
| 12:15 |
miker |
right |
| 12:15 |
* Dyrcona |
was aware that was possible but never did it. |
| 12:15 |
csharp |
ok - I get it |
| 12:15 |
Dyrcona |
We keep everything inside the brick as much as we can. |
| 12:15 |
miker |
it provides HA if you have a service fall over on one brick |
| 12:16 |
miker |
that's the intended purpose, not just a side effect :) |
| 12:17 |
Dyrcona |
Yeah. It makes sense. |
| 12:17 |
Dyrcona |
Actually, I think I did exploit that once to add additional services at MVLC, but that may not have involved a new router, just more drones talking to an existing router. |
| 12:18 |
miker |
csharp: also, http://git.evergreen-ils.org/?p=working/OpenSRF.git;a=shortlog;h=refs/heads/collab/miker/request_queuing-squashed-2018-01-16 may be of interest. request queuing in the face of an out-of-children state |
| 12:19 |
Dyrcona |
miker++ # Someone asked if excess requests were queued last week. |
| 12:19 |
miker |
Dyrcona: it's been used to push brick services into a util router ... not sure if cwmars ever was like that in the past |
| 12:19 |
miker |
yeah, I saw that after the fact (queuing) |
| 12:20 |
Dyrcona |
miker: Presently, C/W MARS runs a separate router on it's util servers. I did consider adding the services to the rest of the brick when I switched the architecture to virtual machines. |
| 12:21 |
Dyrcona |
Same thing for the SIP servers. |
| 12:23 |
csharp |
very interesting |
| 12:26 |
Dyrcona |
Our routers do listen on "public" IPs in the 192.168.X. range with /etc/hosts entries, so it's perfectly doable here. :) |
| 12:27 |
Dyrcona |
That's necessary if you have services running on different servers already. |
| 12:28 |
Dyrcona |
So, just add the other router to the config, restart, and voila! |
| 12:28 |
Dyrcona |
:) |
| 12:30 |
Dyrcona |
Ooh. Belated update, but I found some messages for open-ils.fielder and no children waiting in my logs from the 9th. |
| 12:31 |
Dyrcona |
Am I correct that fielder is only used by the web client? |
| 12:33 |
csharp |
wow - finding 32-bit windows is harder than one would think |
| 12:34 |
* csharp |
finds 32-bit Win 7 Pro ISO in office apps archive |
| 12:38 |
Dyrcona |
heh |
| 12:39 |
Dyrcona |
You're a 32-bit O/S in a 64-bit world. :) |
| 12:39 |
* Dyrcona |
misses the 8-bit days. |
| 12:44 |
dbs |
Last time I saw 32-bit Windows was on our Bibliotheca self-check |
| 12:45 |
* bshum |
stares at it everyday :( |
| 12:45 |
* bshum |
cries inside |
| 12:48 |
|
JBoyer joined #evergreen |
| 12:49 |
JBoyer |
I hope PINES had a successful migration! Everyone on 3.0+ will likely be interested in bug 1743613. |
| 12:49 |
pinesol_green |
Launchpad bug 1743613 in Evergreen "ISBN search only finds records with both 10 and 13 digit ISBNs" [Undecided,New] https://launchpad.net/bugs/1743613 |
| 12:50 |
JBoyer |
It's been a proper treat here recently. |
| 13:02 |
|
khuckins_ joined #evergreen |
| 13:02 |
* JBoyer |
throws smoke pellet |
| 13:03 |
miker |
Dyrcona: it's used by some interfaces in the xul client IIRC, but I bet they're all html :) |
| 13:03 |
miker |
re fielder |
| 13:03 |
miker |
it's not a new services, IOW |
| 13:11 |
Dyrcona |
miker: OK. Thanks. Thought I'd heard of it, but I've never used it in my own work. |
| 13:15 |
pinesol_green |
News from qatests: Failed Running perl live tests <http://testing.evergreen-ils.org/~live> |
| 13:16 |
Dyrcona |
Grr....Lp search is bad. I swear there was a bug to remove the permacrud service. |
| 13:16 |
* csharp |
at least remembers a discussion about that |
| 13:17 |
Dyrcona |
I tried advanced search and even filing a new bug but nothing turned up. |
| 13:18 |
berick |
bug #1680566 |
| 13:18 |
pinesol_green |
Launchpad bug 1680566 in Evergreen "Remove open-ils.permacrud service" [Undecided,New] https://launchpad.net/bugs/1680566 |
| 13:18 |
* berick |
retargets |
| 13:19 |
|
rjackson_isl joined #evergreen |
| 13:19 |
phasefx |
I changed the rss feed for qatester to be more specific. I haven't put in anything yet to prevent a massive spew of errors if something goes wrong early in the build |
| 13:21 |
Dyrcona |
Lp_search-- |
| 13:21 |
bshum |
phasefx++ |
| 13:22 |
Dyrcona |
If I search for just permacrud that bug doesn't turn up. If I search for open-ils.permacrud, it does. |
| 13:23 |
Dyrcona |
I thought the permacrud removal had gone in already. I was looking to verify or deny that. |
| 13:23 |
Dyrcona |
berick++ |
| 13:30 |
kmlussier |
phasefx++ |
| 13:31 |
terran |
Today is the day we find out how many of our libraries still have Windows XP machines in production... |
| 13:32 |
|
khuckins__ joined #evergreen |
| 13:35 |
kmlussier |
terran: :( |
| 13:36 |
* mmorgan |
was always very fond of XP |
| 13:36 |
kmlussier |
On the bright side, at least they aren't using Win98 or ME anymore. |
| 13:37 |
kmlussier |
As I was typing that sentence, I realized that Win98 is 20 years old now. |
| 13:37 |
* kmlussier |
feels old. |
| 13:37 |
berick |
thanks for that kmlussier |
| 13:37 |
kmlussier |
berick: Anytime! |
| 13:38 |
berick |
;) |
| 13:46 |
csharp |
kmlussier: https://www.theonion.com/report-1998-was-ten-fucking-years-ago-1819581939 <-- this was 10 f-ing years ago ;-) |
| 13:48 |
kmlussier |
Ha! |
| 13:48 |
csharp |
I loved The Onion Radio News |
| 13:50 |
kmlussier |
It was also 20 years ago this month that I, along with my shiny new MLS, started working as a librarian. |
| 13:51 |
berick |
kmlussier++ |
| 13:55 |
csharp |
kmlussier++ |
| 13:58 |
terran |
Bmagic: It looks like we are seeing this today https://bugs.launchpad.net/evergreen/+bug/1740537 |
| 13:58 |
pinesol_green |
Launchpad bug 1740537 in Evergreen "Web client: Transit slip printing with wrong information" [Undecided,Confirmed] |
| 13:59 |
Bmagic |
terran: that's awesome! I was beginning to wonder if I had a server problem |
| 13:59 |
Bmagic |
It's not awesome that it's broken, but at least it's not just us |
| 13:59 |
terran |
Bmagic: I wouldn't go so far as to say it's awesome! |
| 14:04 |
csharp |
@who is awesome? |
| 14:04 |
pinesol_green |
ngf42 is awesome. |
| 14:04 |
ngf42 |
LIES! |
| 14:04 |
csharp |
@praise [someone] |
| 14:04 |
* pinesol_green |
In days of old, it was prophesied that a hero would come and restore karmic balance to #evergreen. abneiman is that hero. |
| 14:05 |
csharp |
@blame [someone] |
| 14:05 |
pinesol_green |
gsams broke Evergreen. |
| 14:05 |
ngf42 |
this cat is awesome: https://i.imgur.com/m8EySrW.gifv |
| 14:05 |
berick |
@praise [band] |
| 14:05 |
* pinesol_green |
I come to praise Weird Error 8, not to bury them. |
| 14:05 |
abneiman |
dang, no pressure or anything, pinesol... heh |
| 14:05 |
gsams |
I've been outed! |
| 14:05 |
csharp |
@dunno add https://i.imgur.com/m8EySrW.gifv |
| 14:05 |
pinesol_green |
csharp: The operation succeeded. Dunno #56 added. |
| 14:07 |
csharp |
well, I was able to install and run and print from Hatch successfully on a stock Windows 7 32-bit installation |
| 14:07 |
csharp |
so something else (possibly related to the 32-bit-ness of the workstations) is going on |
| 14:09 |
csharp |
@eightball is it because they're old as hail and need to be replaced? |
| 14:09 |
pinesol_green |
csharp: In your dreams. |
| 14:10 |
berick |
@eightball are you running Win XP? |
| 14:10 |
pinesol_green |
berick: You know the answer better than I. |
| 14:12 |
csharp |
Bmagic: would you be willing to share what you're using to kill off those bad billing queries from bug 1738488? |
| 14:12 |
pinesol_green |
Launchpad bug 1738488 in Evergreen 3.0 "Web client: patron billing history results in long running query" [Medium,Confirmed] https://launchpad.net/bugs/1738488 |
| 14:13 |
csharp |
they're actually not handicapping us at all at the moment, but wanted to be prepared |
| 14:15 |
csharp |
also seeing a huge uptick in the number of these kinds of messages in the logs: |
| 14:15 |
csharp |
Apache2::RequestIO::print: (32) Broken pipe at /usr/lib/x86_64-linux-gnu/perl5/5.22/Template.pm line 180 |
| 14:16 |
berick |
csharp: did you go all-in w/ the browser client? |
| 14:16 |
csharp |
berick: all except catalogers |
| 14:16 |
csharp |
and we suspect some are probably using the xul client anyway |
| 14:16 |
csharp |
(we made it available) |
| 14:17 |
berick |
csharp++ terran++ pines++ # my heroes |
| 14:17 |
berick |
yeah, i'm sure there are some holdouts |
| 14:17 |
csharp |
berick++ # we literally couldn't have done it without you |
| 14:17 |
berick |
@grouphug |
| 14:17 |
pinesol_green |
berick: Try restarting apache. |
| 14:17 |
csharp |
@awww |
| 14:17 |
pinesol_green |
csharp: Message root @ server God....Universe going down for reboot.... |
| 14:18 |
csharp |
we def need an @awww plugin that provides animal gif like the one ngf42 shared :-) |
| 14:18 |
Bmagic |
csharp: I have yet to automate that! |
| 14:18 |
ngf42 |
you want cats? i got cats. |
| 14:19 |
csharp |
Bmagic: ah, so you're just selectively killing them at need? |
| 14:19 |
Bmagic |
csharp: I was just thinking that I need to get that automated before I take off for code4lib |
| 14:19 |
ngf42 |
https://i.imgur.com/hq9zD71.gif |
| 14:19 |
Bmagic |
csharp: yep |
| 14:19 |
csharp |
ngf42++ |
| 14:19 |
ngf42 |
:-) |
| 14:19 |
khuckins__ |
ngf42++ |
| 14:19 |
berick |
Bmagic: yay, i just booked my train for c4l. |
| 14:20 |
Bmagic |
csharp: I figured with the load of your consortium it would be a bigger problem for you which is why I mentioned it strongly the other day |
| 14:20 |
Bmagic |
berick++ |
| 14:20 |
ngf42 |
haha, people are doing actual work and i get karma for cats. i love the internet. |
| 14:20 |
ngf42 |
bmagic++ berick++ csharp++ khuckins__++ |
| 14:21 |
Bmagic |
csharp: https://medium.com/little-programming-joys/finding-and-killing-long-running-queries-on-postgres-7c4f0449e86d |
| 14:22 |
Bmagic |
looks like it will be easy |
| 14:24 |
csharp |
Bmagic: we have a cron in place that gmcharlt helped us with a few years ago: https://pastebin.com/BzQayPUc |
| 14:24 |
pastebot |
"Dyrcona" at 64.57.241.14 pasted "Find long running queries" (10 lines) at http://paste.evergreen-ils.org/968 |
| 14:24 |
Bmagic |
csharp: I seem to recall that you had something in place to kill long running queries? |
| 14:25 |
Bmagic |
yeah, that is close to what I was crafting |
| 14:25 |
csharp |
yeah looks like they're all the same basic approach |
| 14:25 |
pastebot |
"csharp" at 64.57.241.14 pasted "Identify the pids" (5 lines) at http://paste.evergreen-ils.org/969 |
| 14:25 |
Dyrcona |
MVLC had something similar, IIRC, to kill long-running searches. |
| 14:25 |
Bmagic |
but you have to wrap that inside of SELECT pg_cancel_backend(); |
| 14:26 |
Bmagic |
So it might require a DB function with LOOP |
| 14:26 |
Dyrcona |
Bmagic: Don't need a loop, just SELECT pg_cancel_backend(pid).... |
| 14:27 |
Bmagic |
yeah, but what do you do with multiple pids? |
| 14:27 |
Bmagic |
automated |
| 14:27 |
Dyrcona |
Each pid gets cancelled as it comes up, i.e. the function is run once on each row result. |
| 14:29 |
Bmagic |
for example, this is the solution I have for cleaning the auditor tables |
| 14:29 |
Bmagic |
https://github.com/mcoia/mobius_evergreen/tree/master/Random/auditor_table_clean |
| 14:30 |
Dyrcona |
You're looping over tables in that example and not just doing a single query. |
| 14:30 |
Bmagic |
understood, how is killing many pids a single query? Can you run a SELECT inside of the function call? |
| 14:31 |
Dyrcona |
Look at csharp's first paste at pasetbin. |
| 14:31 |
Dyrcona |
That works. |
| 14:31 |
Bmagic |
where is the kill call? |
| 14:32 |
Dyrcona |
Right here: select pg_cancel_backend(pid) from pg_stat_activity where state not like 'idle%' and now() - query_start > '20 seconds' and query ~ 'bib search' order by 1 desc; |
| 14:32 |
Dyrcona |
The pg_cancel_backend() function does it for each pid returned by the from and where. |
| 14:32 |
Bmagic |
I must not be looking at the same paste |
| 14:32 |
Dyrcona |
https://pastebin.com/BzQayPUc |
| 14:32 |
Dyrcona |
That one. |
| 14:33 |
Bmagic |
ah! sorry, I was looking at the wrong thing |
| 14:33 |
Dyrcona |
I pasted something useful, but not the same at about the same time. :) |
| 14:33 |
Bmagic |
ah, I see, cool |
| 14:33 |
Dyrcona |
You could just change the where in csharp's example for the mp.payment query. |
| 14:34 |
Dyrcona |
Make it an or in parenthesis and it would work for both searches and payment history. |
| 14:35 |
pastebot |
"Dyrcona" at 64.57.241.14 pasted "Possible Solution" (4 lines) at http://paste.evergreen-ils.org/970 |
| 14:35 |
Dyrcona |
Yeah, that looks like about right. |
| 14:36 |
Bmagic |
csharp ^^ |
| 14:54 |
csharp |
Bmagic: Dyrcona: cool - thanks |
| 14:58 |
jeffdavis |
This is kinda baffling. I've got a test server with the fix for bug 1736419 where search results include all matching records with located URIs, even when none of the luri's are in scope. |
| 14:58 |
pinesol_green |
Launchpad bug 1736419 in Evergreen "Search Showing Bibs with no Holdings" [High,Fix committed] https://launchpad.net/bugs/1736419 |
| 14:59 |
kmlussier |
jeffdavis: I'm about to submit a bug on this. I don't know if it's the same issue, but we've found that records are being retrieved that are outside of scope when org units are set to not opac visible. |
| 15:00 |
kmlussier |
jeffdavis: I'm still trying to nail down some kind of pattern before submitting the bug. |
| 15:01 |
|
mmorgan1 joined #evergreen |
| 15:10 |
jeffdavis |
kmlussier: Do you mean out-of-scope records are retrieved when they have luri's belonging to opac-invisible org units? |
| 15:12 |
kmlussier |
jeffdavis: Well, that's what I'm trying to nail down. In Concerto, if I made an org unit invisible, I'll start seeing out-of-scope records with luris, but I don't think it's just happening with ones owned by the invisible org unit. |
| 15:13 |
jeffdavis |
Huh, weird. That would be consistent with my test environment which has a number of invisible orgs. |
| 15:15 |
jeffdavis |
I know the bib-level visibility check involves asset.patron_default_visibility_mask() which adds an explicit list of opac-invisible org units to exclude from luri_org vis tests |
| 15:15 |
jeffdavis |
maybe a parenthesis is getting misplaced or something |
| 15:18 |
|
JBoyer joined #evergreen |
| 15:19 |
JBoyer |
csharp, were you the one looking for how to trigger inserts of null user barcodes recently? I was just watching someone renew a card and change barcode and it did it to them. |
| 15:19 |
kmlussier |
OK, when I set BR1 as OPAC invisible, the record with a BR2 luri is being retrieved out of scope. When I set BR2 as OPAC invisible, the record with a BR1 luri is retrieved out of scope. |
| 15:20 |
JBoyer |
What's most annoying is that there was 100% a barcode in the box. We initially gave her 1 card, then changed card numbers again before saving (but didn't save in between). |
| 15:20 |
kmlussier |
When I set SYS1 as OPAC invisible, they both were retrieved out of scope, I think. |
| 15:21 |
JBoyer |
I haven't tried to reproduce it yet (I'm at a new library go live and haven't had time to test much) but wanted to throw this out there in case people want to try to break things. |
| 15:23 |
|
tlittle joined #evergreen |
| 15:26 |
JBoyer |
We got the generic error in database query message (DATABASE_QUERY_ERROR or similar) because PostgreSQL was unhappy, so that's what users will see. |
| 15:27 |
JBoyer |
*poof* |
| 15:42 |
kmlussier |
jeffdavis / mmorgan1: bug 1743650 |
| 15:42 |
pinesol_green |
Launchpad bug 1743650 in Evergreen "Records with Located URIs retrieved out of scope when org units are not OPAC visible" [High,New] https://launchpad.net/bugs/1743650 |
| 15:43 |
* kmlussier |
disappears for a bit. |
| 15:54 |
|
jvwoolf joined #evergreen |
| 16:01 |
|
khuckins_ joined #evergreen |
| 16:02 |
|
mmorgan joined #evergreen |
| 16:56 |
csharp |
hmm - ejabberd is working its ass off on our servers now |
| 16:56 |
csharp |
185% cpu in top |
| 16:56 |
csharp |
I guess that's the upped max children ? |
| 16:58 |
Bmagic |
I think it's going to win! |
| 16:58 |
Bmagic |
Go ejabberd go! |
| 16:58 |
csharp |
:-) |
| 16:58 |
Bmagic |
are you getting errors? |
| 16:59 |
csharp |
nope - logs are clear |
| 16:59 |
Bmagic |
do you have ejabberd centralized and not an instance per brick? |
| 16:59 |
csharp |
all opensrf services are comfortably under their max_children |
| 16:59 |
csharp |
it's per-brick |
| 16:59 |
csharp |
(6 bricks) |
| 16:59 |
Bmagic |
the 185% is all 6 bricks? |
| 16:59 |
csharp |
no - and it's spikes, not constant |
| 17:00 |
csharp |
but it's raising system load above where we normally see it, even on busy days |
| 17:00 |
Bmagic |
yeah, I noticed a higher load on our bricks after 3.0 which I related to the web based staff client being faster and getting more data per second |
| 17:03 |
csharp |
that makes sense |
| 17:03 |
Bmagic |
those ajax calls can ask for alot after the page has loaded |
| 17:04 |
Bmagic |
I love how fast the web based staff client is! The downside (depeding on how you look at it) is that speed comes from somewhere..... application servers/bricks |
| 17:05 |
Bmagic |
depeding/depending |
| 17:05 |
|
mmorgan left #evergreen |
| 17:44 |
ejk |
Funny general question: Any easy way to find max Bib id from an outside process? OpenSRF call, api callback, anything. No biggie. Thanks! |
| 17:51 |
dbs |
ejk: mmm, best I can come up with would be a hardcoded 'psql "SELECT max(id) FROM biblio.record_entry" > /openils/var/web/maxid.html' hack cronjob |
| 17:53 |
ejk |
Heh, I figured that might be. I'm either gonna do a best guess or figure out a direct DB query. Thanks! |
| 17:54 |
dbs |
ejk: ooh, forgot about the 'freshmeat' feed, maybe that could give you a best guess |
| 17:55 |
ejk |
Oh, cool. Learn something new every time I look. =) |
| 17:55 |
dbs |
https://laurentian.concat.ca/opac/extras/feed/freshmeat/atom/biblio/import/10/ for example |
| 17:56 |
dbs |
from http://docs.evergreen-ils.org/2.12/_records.html |
| 17:56 |
dbs |
That works for us because we use bib ID as TCN |
| 17:58 |
dbs |
although actually, <link type="text/html" rel="OPAC" href="http://laurentian.concat.ca/eg/opac/results?query=record_list(3119995,3119994,3119993,3119992,3119991,3119990,3119989,3119988,3119987,3119986)%20sort(edit_date)#descending&amp;locg=1"/> |
| 17:58 |
dbs |
pop the first record off the record_list and that's a pretty good estimate |
| 18:00 |
ejk |
Good stuff. I think I should be able to get a good guess from this. Thanks! |
| 18:00 |
ejk |
dbs++ |
| 18:02 |
dbs |
glad to be able to help |
| 18:32 |
pinesol_green |
News from qatests: Failed Running perl live tests <http://testing.evergreen-ils.org/~live> |
| 19:14 |
jeff |
sounds like that's enough for your needs, but there's also the technique of using pcrud: |
| 19:14 |
jeff |
using srfsh, you could issue: request open-ils.pcrud open-ils.pcrud.id_list.bre "ANONYMOUS", {"id": {"!=": null}}, {"order_by": {"bre":"id desc"}, "limit":1} |
| 19:15 |
jeff |
or the gateway, a slightly ugly url encoded: |
| 19:15 |
jeff |
curl 'https://catalog.tadl.org/osrf-gateway-v1?service=open-ils.pcrud&method=open-ils.pcrud.id_list.bre&param=%22ANONYMOUS%22&param=\{%22id%22:%20\{%22!=%22:%20null\}\}¶m=\{%22order_by%22:%20\{%22bre%22:%22id%20desc%22\},%20%22limit%22:1\}' |
| 19:17 |
jeff |
returns something like: {"payload":[272],"status":200} |
| 19:24 |
jeff |
ejk: are you trying to do something like a locum harvest? |
| 19:39 |
ejk |
jeff: indeed. We are harvesting into elasticsearch. |
