Evergreen ILS Website

IRC log for #evergreen, 2018-09-10

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
00:08 sandbergja joined #evergreen
01:06 beanjammin joined #evergreen
06:31 pinesol News from qatests: Failed Running Evergreen browser client build/test - Expected 6 errors but encountered 3. <http://testing.evergreen-ils.org/~live>
06:31 pinesol News from qatests: Failed Running perl live tests <http://testing.evergreen-ils.org/~live>
06:42 JBoyer joined #evergreen
06:56 agoben joined #evergreen
07:10 rjackson_isl joined #evergreen
07:53 bdljohn joined #evergreen
08:12 csharp seeing segmentation faults running pingest post upgrade
08:12 csharp not sure if it's a data problem on our end or anything to do with the script itself
08:12 csharp current theory is that it's a data problem
08:13 csharp I started it out at 24 concurrent procs (which has worked fine for us on our 64-core server in the past)
08:13 csharp after it segfaulted I dialed it back down to 16 and the same thing happened
08:14 JBoyer how far does it get before dying?
08:14 csharp https://pastebin.com/tgaJacb0 is the error
08:15 csharp JBoyer: I don't have much in the way of metrics - it didn't get through the first 16 batches of 10000
08:17 JBoyer That sounds like a pretty terrible message. Is it just killing that postgres process or the entire server?
08:18 csharp the whole server
08:18 JBoyer I suppose you could throw a RAISE NOTICE in that function and have it dump the current bib id to see if it's a data error.
08:19 JBoyer Ouch. That's really bad.  :(
08:21 JBoyer Have you put that server under fairly significant load to be certain it's not hardware at all?
08:21 JBoyer (That's much more appealing than something in the db being so broken that the entire thing dies....)
08:22 csharp yeah, this is a tried and true test server identical to our prod servers
08:22 csharp been using it since 2014 - all appears to be fine hw-wise
08:22 JBoyer Guess it's a good thing I'm planning to do the same here this week. :/
08:23 csharp I think I'll do a single-thread reingest with some RAISE NOTICEs as you suggested
08:24 JBoyer ++
08:24 JBoyer Good luck
08:24 csharp thanks!
08:24 csharp btw, the upgrade itself was super long (3.0 -> 3.1) because of the billing updates
08:25 csharp gonna be back in here soliciting suggestions for safe old billing removal ;-)
08:26 csharp we have nearly 200 million money.billing rows - I imagine most are for resolved xacts
08:29 JBoyer Seems pretty safe to wipe out anything whose xact is in aged_circulations, but we haven't looked into it here. (that step itself, and each of the fixes I had to apply later because I initially did it wrong, was about 4 hours...)
08:30 JBoyer I'm guessing it's a lot more with 200M rows...
08:34 csharp yeah, we haven't purged any old billings since 2012
08:38 mmorgan joined #evergreen
08:45 JBoyer Oof.
08:48 bos20k joined #evergreen
09:05 yboston joined #evergreen
09:15 lsach joined #evergreen
09:16 Dyrcona joined #evergreen
09:34 jvwoolf joined #evergreen
09:35 terran joined #evergreen
09:49 collum joined #evergreen
09:59 JBoyer Dyrcona, I don't have a lot of great info on the duplicate transits issue, as a test I canceled all of the duplicates and added a unique index on target_copy where the cancel and recv times are null. Haven't heard of any issues since.
10:00 Dyrcona JBoyer: Thanks! That reminds that I need to search the logs for a particular example.
10:03 JBoyer It's clearly still being triggered, as a quick grep of the index name is pulling up 27 attempts this month to insert a dup. :/
10:09 rlefaive joined #evergreen
10:19 csharp 7891c024
10:19 pinesol csharp: [evergreen|Bill Erickson] LP#1750894 Workstation & Cascade settings - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=7891c02>
10:25 mmorgan joined #evergreen
10:32 atheos joined #evergreen
10:34 khuckins joined #evergreen
10:36 terran Happy Bug Squashing Week! agoben++ for being first on the chart
10:36 terran https://docs.google.com/spreadsheets/d/1mDoSEY_f7J​YNyn-sjYwOhB9L8Us4J0Nz_FDcCMf6DQA/edit?usp=sharing
10:36 csharp agoben++
10:38 Dyrcona berick: I'm going to update Lp 1787274 with just the log entries for the copy barcode from the previous spread sheets. It looks like 3 separate checkins were sent within that time frame. I have 6 log entries for the checkin with 3 different session/traces.
10:38 pinesol Launchpad bug 1787274 in Evergreen "Web Client: Transits Don't Always Clear" [Critical,Confirmed] https://launchpad.net/bugs/1787274
10:39 berick Dyrcona: awesome.  and that's interesting, the src_send_times were so close, I assumed it was one xact, but clearly that's a faulty assumption
10:39 berick multiple browser calls would make more sense, sence that's the new code...
10:39 Dyrcona If you want more information, I can share it.
10:43 Dyrcona The authtoken in the logs is defunct, so I didn't bother to scrub it.
10:44 berick thanks
10:48 berick i should really know this...  anyone know if EG 3.2 will work with opensrf 2.5?
10:48 Dyrcona berick: It won't.
10:48 Dyrcona Evergree 3.0+ requires OpenSRF 3.0.
10:49 Dyrcona "It won't," should be "It shouln't." :)
10:49 * Dyrcona can't type this morning, apparently.
10:50 berick made your point either way ;)
10:50 berick thanks Dyrcona
10:51 stephengwills where do I select which circulation matrix weight set to use?
10:58 Dyrcona stephengwills: The database table is config.weight_assoc. It configures which weights are used with which org units. It probably appears in the admin interface somewhere, but I've never really used it.
11:00 rlefaive joined #evergreen
11:06 berick could someone w/ EG blog access please give this a quick poofread?  https://evergreen-ils.org/?p=5449&amp;preview=true
11:08 bshum berick: Reads okay to me
11:08 berick thanks, bshum
11:08 bshum berick++
11:20 stephengwills joined #evergreen
11:27 khuckins_ joined #evergreen
11:29 bos20k_ joined #evergreen
11:29 idjit joined #evergreen
11:39 mmorgan1 joined #evergreen
11:49 rlefaive joined #evergreen
12:00 agoben joined #evergreen
12:00 JBoyer joined #evergreen
12:04 beanjammin joined #evergreen
12:15 jihpringle joined #evergreen
13:06 khuckins joined #evergreen
13:07 jvwoolf joined #evergreen
13:18 bdljohn joined #evergreen
13:22 csharp okay, troubleshooting an issue with copy templates in 3.0.2-ish - similar to bug 1788680, I'm seeing templates with a stat_cat value of "-1"
13:22 pinesol Launchpad bug 1788680 in Evergreen "Null statcats break copy templates" [Undecided,New] https://launchpad.net/bugs/1788680
13:22 csharp so not "null", but still probably a problem
13:22 csharp unless EG considers a value of "-1" ok?
13:24 csharp these are templates auto-ugraded from XUL templates
13:24 csharp upgraded even
13:29 mmorgan csharp: Can't say if it's ok, but fwiw, we have lots of xul templates in our production system with -1 values: {"value":"-1","type":"stat_cat","field":"12"}
13:33 csharp ok - thanks for the confirmation
13:34 csharp by "ok" I mean "not obviously broken", I guess :-)
13:35 jeff "An entry_id of < 0 signifies the stat cat is being removed." -- comment in Open-ILS/xul/staff_client/​server/cat/copy_editor.js
13:55 csharp jeff++ # thanks
14:01 sandbergja joined #evergreen
14:04 sandbergja In the item service, it looks like location is listed twice in the flesh array: https://github.com/evergreen-library-sys​tem/Evergreen/blob/1e6cbaca69712d708bf4e​720e14ac78e1dc24ab6/Open-ILS/web/js/ui/d​efault/staff/circ/services/item.js#L18
14:04 pinesol sandbergja: [evergreen|Dan Wells] LP#1777675 Stamping upgrade script for latest inventory date support - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=1e6cbac>
14:05 sandbergja Is it safe to remove the second occurrence of 'location'?  Nothing broke with my very limited testing...
14:13 pastebot "csharp" at 64.57.241.14 pasted "record that breaks postgres when reingested" (20 lines) at http://paste.evergreen-ils.org/14087
14:13 berick sandbergja: yes, good to remove dupes
14:14 sandbergja berick++
14:14 sandbergja thanks!
14:14 csharp for the curious, when I issue 'update biblio.record_entry set id = id where id = 1979246;' (the record in the paste), postgres crashes
14:15 csharp not sure if that's the only one, but it definitely causes the issue
14:16 csharp 2018-09-10 11:17:19 next-brick01-head open-ils.cstore: [ERR :20475:oils_sql.c:6570:15365876702040844] open-ils.cstore ERROR updating biblio::record_entry object with id = 1979246: 0 SSL SYSCALL error: EOF detect
14:16 csharp ed
14:18 pastebot "csharp" at 64.57.241.14 pasted "error in postgres log" (12 lines) at http://paste.evergreen-ils.org/14088
14:19 csharp the log is truncated after part [11-12]
14:21 rlefaive joined #evergreen
14:26 gmcharlt csharp: two thoughts of possibilities
14:26 Dyrcona csharp: It may be truncated because of some log line size limit.
14:26 gmcharlt (a) that particular record has a (unusual) XML structure error
14:26 gmcharlt (b) issue with an xpath expression
14:27 gmcharlt does this seem to affect more than just this particular record?
14:27 gmcharlt and have you added any indexing definitions recently?
14:28 csharp I am still trying to discern which records are causing reingest to fail, so I don't have enough data yet to know
14:28 jeff fwiw, the marc field from the full paste of the record (psql extended view) seems to parse without warning with xmllint and yaz-marcdump. didn't try throwing it against anything else, like MARC::Record yet.
14:28 csharp I haven't added any indexes - this is on our freshly upgraded 3.2 server
14:29 Dyrcona You said update.... Is the MARC being changed?
14:29 csharp shouldn't be
14:29 csharp oh, well in this case, it was an overlay I think
14:29 csharp as in one of our catalogers attempted to overlay it
14:30 Dyrcona Right, but you're just updating to trigger an ingest.
14:30 csharp but whatever was breaking when running pingest earlier had no manual intervention
14:30 jeff okay, so this could be a distinct issue.
14:30 jeff (still, similar)
14:31 jeff and since the record was being overlaid, we might not have the full content of the marc that was being overlaid.
14:31 jeff and the problem *could* be with the new marc data.
14:32 collum joined #evergreen
14:33 Dyrcona Maybe there was a Ctrl-d in it?
14:33 pastebot "csharp" at 64.57.241.14 pasted "error in postgres log (from pingest run)" (3 lines) at http://paste.evergreen-ils.org/14089
14:33 jeff *sigh*
14:33 jeff Dyrcona: don't remind me... :-)
14:33 csharp ^^ this was during pingest.pl
14:33 csharp eh?
14:33 csharp is that possible?
14:34 * csharp must've missed that one
14:34 jeff csharp: a real life example of something we need to handle for our statewide resource sharing system is an XML message where there is an unescaped Control-D character in a field such as a call number.
14:35 csharp ewww
14:37 csharp c7598ec8
14:37 pinesol csharp: [evergreen|Mike Rylander] LP#1251394: Reingest streamlining, schema realigning, rebasing - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=c7598ec>
14:38 JBoyer Hey csharp, guess what: http://irc.evergreen-ils.org/​evergreen/2018-06-04#i_361991
14:39 JBoyer I thought some of this stuff sounded familiar.
14:39 csharp heh
14:39 csharp it's all a big spiral, leading us back to the tech problems we've all seen before
14:40 JBoyer csharp++ # for finding this stuff on a dev server and not LIVE, like some folks we won't mention today.
14:41 csharp aw jeez - I even branched the solution
14:41 csharp csharp--
14:41 csharp well, possibly, I thought I applied it to our prod server before
14:42 * JBoyer knows how that feels. ;)
14:44 csharp format            | text    | not null default 'mods32'::text                                   | extended
14:44 csharp indeed, that is the issue
14:44 csharp JBoyer++ # memory
14:44 atheos joined #evergreen
14:44 csharp thanks to all who answered :-)
14:47 Dyrcona We stop using grunt after 3.0, right?
14:48 JBoyer Fun fact: the first time I did a google search for my name on the Eg IRC site it automatically turned on SafeSearch. Harsh.
14:49 berick JBoyer is not safe for work
14:49 Dyrcona :)
14:50 Dyrcona To answer my own question, "Yes."
14:50 JBoyer Anyway, without wanting to be "that guy" too badly, this is why I hate running rando-code inside a database process. Hate it. I'm not volunteering to tear it all out and re-architect everything to avoid it so I'm just venting, but still.
14:50 JBoyer berick++
14:52 Dyrcona Well if you did, then we'd have the option of using MariaDB.... /me ducks.
14:54 JBoyer Ick. :P You know they have an SQL Server for Linux now, right? ;)
14:55 JBoyer (They being MS, ever the master of the gener-o-vague product names...)
15:00 jihpringle berick: is the string freeze still Sept 19th or did it get pushed with the updated schedule?
15:05 berick jihpringle: sept 19th at the earliest.  possibly later.
15:06 berick if we don't need a beta2, the release candidate may still be cut on the 19th
15:06 jihpringle thanks, we're just trying to figure out if we can get the new strings for the new basket and preferred names features translated in time
15:07 berick jihpringle: gotcha.  keep me posted?  would be great to have, obviously
15:07 jihpringle will do, the rest of the OPAC now has French translations and the self check is also completely translated
15:08 berick jihpringle++ (and translators)
15:25 JBoyer bshum, Dyrcona, was one of you playing with Postgres 10 and Evergreen? I would assume it works but I haven't had time to look at it and am trying to think about when to upgrade from 9.4
15:26 JBoyer (may as well skip ahead as far as is safe to go)
15:27 bshum Oh, apparently I tried to do it at some point
15:27 bshum Cause I see https://bugs.launchpad.net/evergreen/+bug/1730726
15:27 pinesol Launchpad bug 1730726 in Evergreen "Support for PostgreSQL 10" [Undecided,New]
15:27 Dyrcona I haven't actually used a Pg 10 database server, yet.
15:27 bshum Where I indicated a problem creating the database
15:27 bshum It was so long ago, that I have no idea what was wrong or what I was trying back in 2017
15:32 bshum So I guess we've only gotten as far as PG 9.5 or so
15:34 berick we've been running 9.6 for a while now, no issues
15:35 JBoyer Sounds like 9.6 will be the next jump we make, I'm just not sure when. Evergreen upgrade day seems like a poor choice.
15:35 bshum I think I was planning to work on PG10 further whenever we finally figure out Ubuntu 18.04 support
15:36 JBoyer bshum++
15:36 JBoyer berick++
15:36 JBoyer Dyrcona++
15:36 bshum Mostly because I think that's the stock version that ships with Ubuntu 18.04
15:36 Dyrcona Yeah, I kind of want to get OpenSRF working on Ubuntu 18, first.
15:36 * bshum agrees with Dyrcona
15:50 csharp @who is NSFW?
15:50 pinesol rhamby_ is NSFW.
15:50 csharp pinesol: definitely
15:50 pinesol csharp: What do you mean? An African or European swallow?
15:55 Dyrcona @dunno
15:55 pinesol Dyrcona: As great as you are man, you'll never be greater than yourself.
15:55 Dyrcona pinesol: No truer words were ever spoken.
15:55 pinesol Dyrcona: Yeah, well, you know, that's just, like, your opinion, man.
15:56 Dyrcona I think pinesol is sentient.
16:01 mmorgan @decide Is pinesol sentient?
16:01 pinesol mmorgan: go with Is pinesol sentient?
16:01 terran Does anyone in here know offhand if the Evergreen self-check has a built-in staff time out? (Does it use the Staff Login Inactivity Timeout?)
16:02 bshum terran: I think it's "Self Check: Patron Login Timeout (in seconds)" or something like that, right.
16:03 bshum Or at least this really old bug seems to say that we made it work with that once upon a version: https://bugs.launchpad.net/evergreen/+bug/1306814
16:03 pinesol Launchpad bug 1306814 in Evergreen "Self checkout ignoring patron timeout library setting" [Medium,Fix released]
16:03 Dyrcona Yeah, I was just gonna look at the settings.
16:03 Dyrcona I think bshum is correct.
16:03 bshum Or you mean staff timeout like the selfcheck interface will log out if not used byond X time?
16:03 terran bshum: That's it for the patron login, but the staff has to log in first
16:03 terran The latter, yes
16:03 bshum Ah, once initialized by staff, I think it just stays on
16:04 bshum For as long as the browser keeps it around
16:04 bshum Or perhaps until the auth token expires?
16:04 terran I tried leaving it on overnight and it logged out, but I don't know at what time
16:04 * mmorgan is checking for notes, but believes it uses the same for other staff logins.
16:04 bshum Probably
16:06 terran Thanks, we're using the 7200 second timeout for staff so I'm testing to see if it stays on longer than that
16:06 terran (We had one library say that staff keep having to log in.)
16:07 * bshum blames the kiosk
16:07 terran bshum: that's where I'm leaning
16:08 terran I asked them to check their custom settings
16:08 bshum Sometimes the kiosk browser resets their session based on their own timings or cache clearing processes.
16:08 bshum But yeah, lots of little things
16:11 terran Thanks, that all reinforces my thoughts :)
16:11 rhamby_ csharp: it's a fair cop
16:14 Dyrcona :)
16:17 terran berick++ for first patch signed off for bug squashing week! (and thanks to Garry for testing, but I don't see him in here)
16:32 berick woohoo
16:32 berick Garry++
16:39 phasefx berick: have you tried the eg2/ work on wheezy by chance?
16:39 berick phasefx: no, it's been many moons since I've used wheezy.  just remember you have to update nodejs
16:40 bshum Isn't wheezy dead yet?
16:40 Dyrcona I would not expect it to work on Wheezy.
16:40 Dyrcona Yes, wheezy is officially dead since the end of May.
16:41 * Dyrcona wanted to remove it from the prereqs months ago.
16:41 phasefx ah
16:41 Dyrcona There is/are Lp bugs with branches to remove wheezy prereqs if anyone wants to dust them off.
16:41 bshum https://bugs.launchpad.net/evergreen/+bug/1718459
16:41 pinesol Launchpad bug 1718459 in OpenSRF "Remove Installation Targets for Debian Wheezy" [Undecided,New]
16:41 phasefx it'll take me a while to get the live tests moved over to something newer
16:45 jvwoolf left #evergreen
16:59 khuckins_ joined #evergreen
17:02 Christineb joined #evergreen
17:11 mmorgan left #evergreen
17:14 sandbergja joined #evergreen
18:07 sandbergja joined #evergreen
21:21 finnx joined #evergreen
21:21 finnx left #evergreen
22:23 beanjammin joined #evergreen
22:50 beanjammin joined #evergreen
23:21 sandbergja joined #evergreen

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat