| Time |
Nick |
Message |
| 00:08 |
|
sandbergja joined #evergreen |
| 01:06 |
|
beanjammin joined #evergreen |
| 06:31 |
pinesol |
News from qatests: Failed Running Evergreen browser client build/test - Expected 6 errors but encountered 3. <http://testing.evergreen-ils.org/~live> |
| 06:31 |
pinesol |
News from qatests: Failed Running perl live tests <http://testing.evergreen-ils.org/~live> |
| 06:42 |
|
JBoyer joined #evergreen |
| 06:56 |
|
agoben joined #evergreen |
| 07:10 |
|
rjackson_isl joined #evergreen |
| 07:53 |
|
bdljohn joined #evergreen |
| 08:12 |
csharp |
seeing segmentation faults running pingest post upgrade |
| 08:12 |
csharp |
not sure if it's a data problem on our end or anything to do with the script itself |
| 08:12 |
csharp |
current theory is that it's a data problem |
| 08:13 |
csharp |
I started it out at 24 concurrent procs (which has worked fine for us on our 64-core server in the past) |
| 08:13 |
csharp |
after it segfaulted I dialed it back down to 16 and the same thing happened |
| 08:14 |
JBoyer |
how far does it get before dying? |
| 08:14 |
csharp |
https://pastebin.com/tgaJacb0 is the error |
| 08:15 |
csharp |
JBoyer: I don't have much in the way of metrics - it didn't get through the first 16 batches of 10000 |
| 08:17 |
JBoyer |
That sounds like a pretty terrible message. Is it just killing that postgres process or the entire server? |
| 08:18 |
csharp |
the whole server |
| 08:18 |
JBoyer |
I suppose you could throw a RAISE NOTICE in that function and have it dump the current bib id to see if it's a data error. |
| 08:19 |
JBoyer |
Ouch. That's really bad. :( |
| 08:21 |
JBoyer |
Have you put that server under fairly significant load to be certain it's not hardware at all? |
| 08:21 |
JBoyer |
(That's much more appealing than something in the db being so broken that the entire thing dies....) |
| 08:22 |
csharp |
yeah, this is a tried and true test server identical to our prod servers |
| 08:22 |
csharp |
been using it since 2014 - all appears to be fine hw-wise |
| 08:22 |
JBoyer |
Guess it's a good thing I'm planning to do the same here this week. :/ |
| 08:23 |
csharp |
I think I'll do a single-thread reingest with some RAISE NOTICEs as you suggested |
| 08:24 |
JBoyer |
++ |
| 08:24 |
JBoyer |
Good luck |
| 08:24 |
csharp |
thanks! |
| 08:24 |
csharp |
btw, the upgrade itself was super long (3.0 -> 3.1) because of the billing updates |
| 08:25 |
csharp |
gonna be back in here soliciting suggestions for safe old billing removal ;-) |
| 08:26 |
csharp |
we have nearly 200 million money.billing rows - I imagine most are for resolved xacts |
| 08:29 |
JBoyer |
Seems pretty safe to wipe out anything whose xact is in aged_circulations, but we haven't looked into it here. (that step itself, and each of the fixes I had to apply later because I initially did it wrong, was about 4 hours...) |
| 08:30 |
JBoyer |
I'm guessing it's a lot more with 200M rows... |
| 08:34 |
csharp |
yeah, we haven't purged any old billings since 2012 |
| 08:38 |
|
mmorgan joined #evergreen |
| 08:45 |
JBoyer |
Oof. |
| 08:48 |
|
bos20k joined #evergreen |
| 09:05 |
|
yboston joined #evergreen |
| 09:15 |
|
lsach joined #evergreen |
| 09:16 |
|
Dyrcona joined #evergreen |
| 09:34 |
|
jvwoolf joined #evergreen |
| 09:35 |
|
terran joined #evergreen |
| 09:49 |
|
collum joined #evergreen |
| 09:59 |
JBoyer |
Dyrcona, I don't have a lot of great info on the duplicate transits issue, as a test I canceled all of the duplicates and added a unique index on target_copy where the cancel and recv times are null. Haven't heard of any issues since. |
| 10:00 |
Dyrcona |
JBoyer: Thanks! That reminds that I need to search the logs for a particular example. |
| 10:03 |
JBoyer |
It's clearly still being triggered, as a quick grep of the index name is pulling up 27 attempts this month to insert a dup. :/ |
| 10:09 |
|
rlefaive joined #evergreen |
| 10:19 |
csharp |
7891c024 |
| 10:19 |
pinesol |
csharp: [evergreen|Bill Erickson] LP#1750894 Workstation & Cascade settings - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=7891c02> |
| 10:25 |
|
mmorgan joined #evergreen |
| 10:32 |
|
atheos joined #evergreen |
| 10:34 |
|
khuckins joined #evergreen |
| 10:36 |
terran |
Happy Bug Squashing Week! agoben++ for being first on the chart |
| 10:36 |
terran |
https://docs.google.com/spreadsheets/d/1mDoSEY_f7JYNyn-sjYwOhB9L8Us4J0Nz_FDcCMf6DQA/edit?usp=sharing |
| 10:36 |
csharp |
agoben++ |
| 10:38 |
Dyrcona |
berick: I'm going to update Lp 1787274 with just the log entries for the copy barcode from the previous spread sheets. It looks like 3 separate checkins were sent within that time frame. I have 6 log entries for the checkin with 3 different session/traces. |
| 10:38 |
pinesol |
Launchpad bug 1787274 in Evergreen "Web Client: Transits Don't Always Clear" [Critical,Confirmed] https://launchpad.net/bugs/1787274 |
| 10:39 |
berick |
Dyrcona: awesome. and that's interesting, the src_send_times were so close, I assumed it was one xact, but clearly that's a faulty assumption |
| 10:39 |
berick |
multiple browser calls would make more sense, sence that's the new code... |
| 10:39 |
Dyrcona |
If you want more information, I can share it. |
| 10:43 |
Dyrcona |
The authtoken in the logs is defunct, so I didn't bother to scrub it. |
| 10:44 |
berick |
thanks |
| 10:48 |
berick |
i should really know this... anyone know if EG 3.2 will work with opensrf 2.5? |
| 10:48 |
Dyrcona |
berick: It won't. |
| 10:48 |
Dyrcona |
Evergree 3.0+ requires OpenSRF 3.0. |
| 10:49 |
Dyrcona |
"It won't," should be "It shouln't." :) |
| 10:49 |
* Dyrcona |
can't type this morning, apparently. |
| 10:50 |
berick |
made your point either way ;) |
| 10:50 |
berick |
thanks Dyrcona |
| 10:51 |
stephengwills |
where do I select which circulation matrix weight set to use? |
| 10:58 |
Dyrcona |
stephengwills: The database table is config.weight_assoc. It configures which weights are used with which org units. It probably appears in the admin interface somewhere, but I've never really used it. |
| 11:00 |
|
rlefaive joined #evergreen |
| 11:06 |
berick |
could someone w/ EG blog access please give this a quick poofread? https://evergreen-ils.org/?p=5449&preview=true |
| 11:08 |
bshum |
berick: Reads okay to me |
| 11:08 |
berick |
thanks, bshum |
| 11:08 |
bshum |
berick++ |
| 11:20 |
|
stephengwills joined #evergreen |
| 11:27 |
|
khuckins_ joined #evergreen |
| 11:29 |
|
bos20k_ joined #evergreen |
| 11:29 |
|
idjit joined #evergreen |
| 11:39 |
|
mmorgan1 joined #evergreen |
| 11:49 |
|
rlefaive joined #evergreen |
| 12:00 |
|
agoben joined #evergreen |
| 12:00 |
|
JBoyer joined #evergreen |
| 12:04 |
|
beanjammin joined #evergreen |
| 12:15 |
|
jihpringle joined #evergreen |
| 13:06 |
|
khuckins joined #evergreen |
| 13:07 |
|
jvwoolf joined #evergreen |
| 13:18 |
|
bdljohn joined #evergreen |
| 13:22 |
csharp |
okay, troubleshooting an issue with copy templates in 3.0.2-ish - similar to bug 1788680, I'm seeing templates with a stat_cat value of "-1" |
| 13:22 |
pinesol |
Launchpad bug 1788680 in Evergreen "Null statcats break copy templates" [Undecided,New] https://launchpad.net/bugs/1788680 |
| 13:22 |
csharp |
so not "null", but still probably a problem |
| 13:22 |
csharp |
unless EG considers a value of "-1" ok? |
| 13:24 |
csharp |
these are templates auto-ugraded from XUL templates |
| 13:24 |
csharp |
upgraded even |
| 13:29 |
mmorgan |
csharp: Can't say if it's ok, but fwiw, we have lots of xul templates in our production system with -1 values: {"value":"-1","type":"stat_cat","field":"12"} |
| 13:33 |
csharp |
ok - thanks for the confirmation |
| 13:34 |
csharp |
by "ok" I mean "not obviously broken", I guess :-) |
| 13:35 |
jeff |
"An entry_id of < 0 signifies the stat cat is being removed." -- comment in Open-ILS/xul/staff_client/server/cat/copy_editor.js |
| 13:55 |
csharp |
jeff++ # thanks |
| 14:01 |
|
sandbergja joined #evergreen |
| 14:04 |
sandbergja |
In the item service, it looks like location is listed twice in the flesh array: https://github.com/evergreen-library-system/Evergreen/blob/1e6cbaca69712d708bf4e720e14ac78e1dc24ab6/Open-ILS/web/js/ui/default/staff/circ/services/item.js#L18 |
| 14:04 |
pinesol |
sandbergja: [evergreen|Dan Wells] LP#1777675 Stamping upgrade script for latest inventory date support - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=1e6cbac> |
| 14:05 |
sandbergja |
Is it safe to remove the second occurrence of 'location'? Nothing broke with my very limited testing... |
| 14:13 |
pastebot |
"csharp" at 64.57.241.14 pasted "record that breaks postgres when reingested" (20 lines) at http://paste.evergreen-ils.org/14087 |
| 14:13 |
berick |
sandbergja: yes, good to remove dupes |
| 14:14 |
sandbergja |
berick++ |
| 14:14 |
sandbergja |
thanks! |
| 14:14 |
csharp |
for the curious, when I issue 'update biblio.record_entry set id = id where id = 1979246;' (the record in the paste), postgres crashes |
| 14:15 |
csharp |
not sure if that's the only one, but it definitely causes the issue |
| 14:16 |
csharp |
2018-09-10 11:17:19 next-brick01-head open-ils.cstore: [ERR :20475:oils_sql.c:6570:15365876702040844] open-ils.cstore ERROR updating biblio::record_entry object with id = 1979246: 0 SSL SYSCALL error: EOF detect |
| 14:16 |
csharp |
ed |
| 14:18 |
pastebot |
"csharp" at 64.57.241.14 pasted "error in postgres log" (12 lines) at http://paste.evergreen-ils.org/14088 |
| 14:19 |
csharp |
the log is truncated after part [11-12] |
| 14:21 |
|
rlefaive joined #evergreen |
| 14:26 |
gmcharlt |
csharp: two thoughts of possibilities |
| 14:26 |
Dyrcona |
csharp: It may be truncated because of some log line size limit. |
| 14:26 |
gmcharlt |
(a) that particular record has a (unusual) XML structure error |
| 14:26 |
gmcharlt |
(b) issue with an xpath expression |
| 14:27 |
gmcharlt |
does this seem to affect more than just this particular record? |
| 14:27 |
gmcharlt |
and have you added any indexing definitions recently? |
| 14:28 |
csharp |
I am still trying to discern which records are causing reingest to fail, so I don't have enough data yet to know |
| 14:28 |
jeff |
fwiw, the marc field from the full paste of the record (psql extended view) seems to parse without warning with xmllint and yaz-marcdump. didn't try throwing it against anything else, like MARC::Record yet. |
| 14:28 |
csharp |
I haven't added any indexes - this is on our freshly upgraded 3.2 server |
| 14:29 |
Dyrcona |
You said update.... Is the MARC being changed? |
| 14:29 |
csharp |
shouldn't be |
| 14:29 |
csharp |
oh, well in this case, it was an overlay I think |
| 14:29 |
csharp |
as in one of our catalogers attempted to overlay it |
| 14:30 |
Dyrcona |
Right, but you're just updating to trigger an ingest. |
| 14:30 |
csharp |
but whatever was breaking when running pingest earlier had no manual intervention |
| 14:30 |
jeff |
okay, so this could be a distinct issue. |
| 14:30 |
jeff |
(still, similar) |
| 14:31 |
jeff |
and since the record was being overlaid, we might not have the full content of the marc that was being overlaid. |
| 14:31 |
jeff |
and the problem *could* be with the new marc data. |
| 14:32 |
|
collum joined #evergreen |
| 14:33 |
Dyrcona |
Maybe there was a Ctrl-d in it? |
| 14:33 |
pastebot |
"csharp" at 64.57.241.14 pasted "error in postgres log (from pingest run)" (3 lines) at http://paste.evergreen-ils.org/14089 |
| 14:33 |
jeff |
*sigh* |
| 14:33 |
jeff |
Dyrcona: don't remind me... :-) |
| 14:33 |
csharp |
^^ this was during pingest.pl |
| 14:33 |
csharp |
eh? |
| 14:33 |
csharp |
is that possible? |
| 14:34 |
* csharp |
must've missed that one |
| 14:34 |
jeff |
csharp: a real life example of something we need to handle for our statewide resource sharing system is an XML message where there is an unescaped Control-D character in a field such as a call number. |
| 14:35 |
csharp |
ewww |
| 14:37 |
csharp |
c7598ec8 |
| 14:37 |
pinesol |
csharp: [evergreen|Mike Rylander] LP#1251394: Reingest streamlining, schema realigning, rebasing - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=c7598ec> |
| 14:38 |
JBoyer |
Hey csharp, guess what: http://irc.evergreen-ils.org/evergreen/2018-06-04#i_361991 |
| 14:39 |
JBoyer |
I thought some of this stuff sounded familiar. |
| 14:39 |
csharp |
heh |
| 14:39 |
csharp |
it's all a big spiral, leading us back to the tech problems we've all seen before |
| 14:40 |
JBoyer |
csharp++ # for finding this stuff on a dev server and not LIVE, like some folks we won't mention today. |
| 14:41 |
csharp |
aw jeez - I even branched the solution |
| 14:41 |
csharp |
csharp-- |
| 14:41 |
csharp |
well, possibly, I thought I applied it to our prod server before |
| 14:42 |
* JBoyer |
knows how that feels. ;) |
| 14:44 |
csharp |
format | text | not null default 'mods32'::text | extended |
| 14:44 |
csharp |
indeed, that is the issue |
| 14:44 |
csharp |
JBoyer++ # memory |
| 14:44 |
|
atheos joined #evergreen |
| 14:44 |
csharp |
thanks to all who answered :-) |
| 14:47 |
Dyrcona |
We stop using grunt after 3.0, right? |
| 14:48 |
JBoyer |
Fun fact: the first time I did a google search for my name on the Eg IRC site it automatically turned on SafeSearch. Harsh. |
| 14:49 |
berick |
JBoyer is not safe for work |
| 14:49 |
Dyrcona |
:) |
| 14:50 |
Dyrcona |
To answer my own question, "Yes." |
| 14:50 |
JBoyer |
Anyway, without wanting to be "that guy" too badly, this is why I hate running rando-code inside a database process. Hate it. I'm not volunteering to tear it all out and re-architect everything to avoid it so I'm just venting, but still. |
| 14:50 |
JBoyer |
berick++ |
| 14:52 |
Dyrcona |
Well if you did, then we'd have the option of using MariaDB.... /me ducks. |
| 14:54 |
JBoyer |
Ick. :P You know they have an SQL Server for Linux now, right? ;) |
| 14:55 |
JBoyer |
(They being MS, ever the master of the gener-o-vague product names...) |
| 15:00 |
jihpringle |
berick: is the string freeze still Sept 19th or did it get pushed with the updated schedule? |
| 15:05 |
berick |
jihpringle: sept 19th at the earliest. possibly later. |
| 15:06 |
berick |
if we don't need a beta2, the release candidate may still be cut on the 19th |
| 15:06 |
jihpringle |
thanks, we're just trying to figure out if we can get the new strings for the new basket and preferred names features translated in time |
| 15:07 |
berick |
jihpringle: gotcha. keep me posted? would be great to have, obviously |
| 15:07 |
jihpringle |
will do, the rest of the OPAC now has French translations and the self check is also completely translated |
| 15:08 |
berick |
jihpringle++ (and translators) |
| 15:25 |
JBoyer |
bshum, Dyrcona, was one of you playing with Postgres 10 and Evergreen? I would assume it works but I haven't had time to look at it and am trying to think about when to upgrade from 9.4 |
| 15:26 |
JBoyer |
(may as well skip ahead as far as is safe to go) |
| 15:27 |
bshum |
Oh, apparently I tried to do it at some point |
| 15:27 |
bshum |
Cause I see https://bugs.launchpad.net/evergreen/+bug/1730726 |
| 15:27 |
pinesol |
Launchpad bug 1730726 in Evergreen "Support for PostgreSQL 10" [Undecided,New] |
| 15:27 |
Dyrcona |
I haven't actually used a Pg 10 database server, yet. |
| 15:27 |
bshum |
Where I indicated a problem creating the database |
| 15:27 |
bshum |
It was so long ago, that I have no idea what was wrong or what I was trying back in 2017 |
| 15:32 |
bshum |
So I guess we've only gotten as far as PG 9.5 or so |
| 15:34 |
berick |
we've been running 9.6 for a while now, no issues |
| 15:35 |
JBoyer |
Sounds like 9.6 will be the next jump we make, I'm just not sure when. Evergreen upgrade day seems like a poor choice. |
| 15:35 |
bshum |
I think I was planning to work on PG10 further whenever we finally figure out Ubuntu 18.04 support |
| 15:36 |
JBoyer |
bshum++ |
| 15:36 |
JBoyer |
berick++ |
| 15:36 |
JBoyer |
Dyrcona++ |
| 15:36 |
bshum |
Mostly because I think that's the stock version that ships with Ubuntu 18.04 |
| 15:36 |
Dyrcona |
Yeah, I kind of want to get OpenSRF working on Ubuntu 18, first. |
| 15:36 |
* bshum |
agrees with Dyrcona |
| 15:50 |
csharp |
@who is NSFW? |
| 15:50 |
pinesol |
rhamby_ is NSFW. |
| 15:50 |
csharp |
pinesol: definitely |
| 15:50 |
pinesol |
csharp: What do you mean? An African or European swallow? |
| 15:55 |
Dyrcona |
@dunno |
| 15:55 |
pinesol |
Dyrcona: As great as you are man, you'll never be greater than yourself. |
| 15:55 |
Dyrcona |
pinesol: No truer words were ever spoken. |
| 15:55 |
pinesol |
Dyrcona: Yeah, well, you know, that's just, like, your opinion, man. |
| 15:56 |
Dyrcona |
I think pinesol is sentient. |
| 16:01 |
mmorgan |
@decide Is pinesol sentient? |
| 16:01 |
pinesol |
mmorgan: go with Is pinesol sentient? |
| 16:01 |
terran |
Does anyone in here know offhand if the Evergreen self-check has a built-in staff time out? (Does it use the Staff Login Inactivity Timeout?) |
| 16:02 |
bshum |
terran: I think it's "Self Check: Patron Login Timeout (in seconds)" or something like that, right. |
| 16:03 |
bshum |
Or at least this really old bug seems to say that we made it work with that once upon a version: https://bugs.launchpad.net/evergreen/+bug/1306814 |
| 16:03 |
pinesol |
Launchpad bug 1306814 in Evergreen "Self checkout ignoring patron timeout library setting" [Medium,Fix released] |
| 16:03 |
Dyrcona |
Yeah, I was just gonna look at the settings. |
| 16:03 |
Dyrcona |
I think bshum is correct. |
| 16:03 |
bshum |
Or you mean staff timeout like the selfcheck interface will log out if not used byond X time? |
| 16:03 |
terran |
bshum: That's it for the patron login, but the staff has to log in first |
| 16:03 |
terran |
The latter, yes |
| 16:03 |
bshum |
Ah, once initialized by staff, I think it just stays on |
| 16:04 |
bshum |
For as long as the browser keeps it around |
| 16:04 |
bshum |
Or perhaps until the auth token expires? |
| 16:04 |
terran |
I tried leaving it on overnight and it logged out, but I don't know at what time |
| 16:04 |
* mmorgan |
is checking for notes, but believes it uses the same for other staff logins. |
| 16:04 |
bshum |
Probably |
| 16:06 |
terran |
Thanks, we're using the 7200 second timeout for staff so I'm testing to see if it stays on longer than that |
| 16:06 |
terran |
(We had one library say that staff keep having to log in.) |
| 16:07 |
* bshum |
blames the kiosk |
| 16:07 |
terran |
bshum: that's where I'm leaning |
| 16:08 |
terran |
I asked them to check their custom settings |
| 16:08 |
bshum |
Sometimes the kiosk browser resets their session based on their own timings or cache clearing processes. |
| 16:08 |
bshum |
But yeah, lots of little things |
| 16:11 |
terran |
Thanks, that all reinforces my thoughts :) |
| 16:11 |
rhamby_ |
csharp: it's a fair cop |
| 16:14 |
Dyrcona |
:) |
| 16:17 |
terran |
berick++ for first patch signed off for bug squashing week! (and thanks to Garry for testing, but I don't see him in here) |
| 16:32 |
berick |
woohoo |
| 16:32 |
berick |
Garry++ |
| 16:39 |
phasefx |
berick: have you tried the eg2/ work on wheezy by chance? |
| 16:39 |
berick |
phasefx: no, it's been many moons since I've used wheezy. just remember you have to update nodejs |
| 16:40 |
bshum |
Isn't wheezy dead yet? |
| 16:40 |
Dyrcona |
I would not expect it to work on Wheezy. |
| 16:40 |
Dyrcona |
Yes, wheezy is officially dead since the end of May. |
| 16:41 |
* Dyrcona |
wanted to remove it from the prereqs months ago. |
| 16:41 |
phasefx |
ah |
| 16:41 |
Dyrcona |
There is/are Lp bugs with branches to remove wheezy prereqs if anyone wants to dust them off. |
| 16:41 |
bshum |
https://bugs.launchpad.net/evergreen/+bug/1718459 |
| 16:41 |
pinesol |
Launchpad bug 1718459 in OpenSRF "Remove Installation Targets for Debian Wheezy" [Undecided,New] |
| 16:41 |
phasefx |
it'll take me a while to get the live tests moved over to something newer |
| 16:45 |
|
jvwoolf left #evergreen |
| 16:59 |
|
khuckins_ joined #evergreen |
| 17:02 |
|
Christineb joined #evergreen |
| 17:11 |
|
mmorgan left #evergreen |
| 17:14 |
|
sandbergja joined #evergreen |
| 18:07 |
|
sandbergja joined #evergreen |
| 21:21 |
|
finnx joined #evergreen |
| 21:21 |
|
finnx left #evergreen |
| 22:23 |
|
beanjammin joined #evergreen |
| 22:50 |
|
beanjammin joined #evergreen |
| 23:21 |
|
sandbergja joined #evergreen |
