Evergreen ILS Website

IRC log for #evergreen, 2021-03-11

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
00:41 dbwells joined #evergreen
00:59 sandbergja joined #evergreen
01:10 malexander joined #evergreen
01:55 dbwells joined #evergreen
02:40 dbwells joined #evergreen
03:26 dbwells joined #evergreen
04:12 dbwells joined #evergreen
04:41 malexander joined #evergreen
04:48 dbwells joined #evergreen
05:40 dbwells joined #evergreen
06:01 pinesol News from qatests: Testing Success <http://testing.evergreen-ils.org/~live>
06:09 dbwells joined #evergreen
07:16 rjackson_isl_hom joined #evergreen
08:03 mantis1 joined #evergreen
08:05 phasefx_ joined #evergreen
08:06 jeff__ joined #evergreen
08:06 genpaku_ joined #evergreen
08:07 abneiman joined #evergreen
08:07 egbuilder joined #evergreen
08:11 yeats joined #evergreen
08:12 collum joined #evergreen
08:13 malexander joined #evergreen
08:21 rfrasur joined #evergreen
08:29 mmorgan joined #evergreen
08:35 csharp miker: unfortunately, the sideloader consumed all available RAM and swap on the server and it locked up (64GB/8 core testing standalone DB server)
08:35 csharp miker: last message from the script was Dictionary built in 9840 seconds, writing..
08:54 jvwoolf joined #evergreen
09:04 miker csharp: that's ... very strange. once it starts writing the file, it should start /decreasing/ in ram use because it starts deleting keys from the huge hash. I have not seen any other behavior in my testing :(
09:06 miker csharp: curious what version of perl is on that server...
09:06 Dyrcona joined #evergreen
09:09 alynn26 joined #evergreen
09:23 csharp miker: 5.26.1
09:23 csharp Ubuntu 18.04
09:27 miker I have 5.28 (kubuntu 19.10), though I certainly can't pin it on that with any confidence. I have an idea for a memory-related tweak, if you're willing to test again in a couple hours (meetings!).  even if the release folken would rather not grant an indulgence, I'd appreciate having it ready to merge immediately after 3.7 is branched -- I'm tired of these bigger ones not getting in at the wire ;)
09:27 csharp yes, totally willing to test
09:31 Dyrcona Ubuntu 20.04 has Perl 5.30. I have an Ubuntu 20.04 dvelopment VM and DB server. The latter has 128GB of RAM.
09:32 csharp our staging DB used to be a clone (specs-wise) of our prod DB, but in our new environment, we've been required to slim down our boxen based on usage
09:33 csharp and that DB doesn't get a lot of day-to-day use
09:33 dbwells joined #evergreen
09:33 Dyrcona My development DB server is an old mail server with 128GB of RAM and 8TB of disk space, so I run multiple versions of Pg with multiple copies of production data.
09:34 dbwells joined #evergreen
09:34 Dyrcona I'm trying to find the bug number in yesterday's log.
09:35 miker csharp: fwiw, I suspect it's the number of suggestions on very short prefix keys, especially coming from the "keyword" data.  there are runtime rules about what suggestions can be ignored, and I'm going to cull suggestions that wouldn't pass that test. the only way I can think of memory increasing during the write phase is if there are 10s of thousands of suggestions on a key (and even then, we should be talking MB, not GB, of RAM)... but we shall see
09:36 Dyrcona OK. Got the branch. I'll take a look just to see how it works for us.We have 2.3 million not deleted bibs.
09:41 Dyrcona Um. Is git.evergreen-ils.org down?
09:44 csharp miker: If I can provide any data from my end, let me know
09:44 csharp Dyrcona: coming up for me
09:44 Dyrcona csharp: See what I said in bofh.
09:45 Dyrcona I must have bad timing. It's working, now.
09:52 Dyrcona csharp++
09:57 miker csharp: was there anything written to the output file? if so, how many lines?
10:00 Dyrcona I've observed two things from looking at the DYM branch so far: 1. It needs a rebase on master and to have dependencies added for Ubuntu 20.04 and 2. the new Perl modules should be listed under CPAN_MODULES_PGSQL since the DB server needs them, too.
10:02 Dyrcona That's based on the ubuntu bionic Makefile.
10:14 csharp miker: 3156816 sideload.sql
10:20 Dyrcona miker csharp: I have more observations as a result of running the code. I'll update the bug by early afternoon.
10:22 csharp miker++ Dyrcona++
10:49 wsmoak joined #evergreen
11:28 gmcharlt noting for the room that the RM today has decided that today is the second coming of 10 March.
11:28 gmcharlt i.e., we extending the freeze through today (and maybe through tomorrow morning)
11:29 gmcharlt but still intend to cut a beta tomorrow
11:30 Dyrcona gmcharlt++
11:30 gmcharlt *RM team
11:36 csharp gmcharlt++
11:45 malexander joined #evergreen
12:04 miker csharp: I assume the end of the output file is not "\."?
12:04 miker and gmcharlt++
12:07 jihpringle joined #evergreen
12:34 collum joined #evergreen
12:39 terranm joined #evergreen
12:41 collum joined #evergreen
12:42 jeffdavis Looks like our action_trigger_runner.pl has been hanging since yesterday afternoon, is it "safe" to kill the process or is there a better way to handle it?
12:47 JBoyer jeffdavis, you might have to reset some events depending on what it was doing but there shouldn't be any harm in knocking it over; it's probably not actually doing anything anymore.
12:50 jeffdavis Thanks!
12:52 Dyrcona "Disappointment Number (Unknown)" :)
12:57 jihpringle joined #evergreen
13:09 khuckins joined #evergreen
13:13 malexander joined #evergreen
13:45 * gmcharlt claims 1254
13:56 pinesol Showing latest 5 of 8 commits to Evergreen...
13:56 pinesol [evergreen|Mike Rylander] Provide two library groups for testing with Concerto - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=2fcec1d>
13:56 pinesol [evergreen|Mike Rylander] LP#1815815: Library Groups - limit copy table - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=a9e0656>
13:56 pinesol [evergreen|Mike Rylander] LP#1815815: Do not use depth when filtering copies for a library group - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=76f53ef>
13:56 pinesol [evergreen|Galen Charlton] LP#1815815: add release notes - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=0bf0441>
13:56 pinesol [evergreen|Galen Charlton] LP#1815815: stamp schema update - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=e98a375>
13:58 terranm joined #evergreen
14:13 csharp miker: last line is:
14:13 csharp yu20\N0{nyug2941756b}1{yu2}0\N0\N0\N
14:18 terranm joined #evergreen
14:18 jeffdavis tempted to @band add that string
14:19 berick @ana yu2I0I\NI0I{nyug2941756b}I1I{yu2}I0I\NI0I\NI0I\N
14:19 pinesol berick: I I, Yi in I buy in guy
14:20 jeffdavis :)
14:26 collum Anyone know the sms gateway for SafetyNet Wireless?
14:30 collum It's possibly sprint.  I will see if the patron is willing to experiment.
14:34 Dyrcona @ana safetynet wireless
14:34 pinesol Dyrcona: Serenely wisest fat
14:34 Dyrcona :)
14:35 Dyrcona symspell-sideload.pl is still readiing input on my db server after 176 minutes and using 74GB of RAM so far.
14:36 Dyrcona It's up to 85,740,000 lines out of 103,708,944.
14:38 rhamby collum : yeah safetynet is a sprint mvno, I haven't had them but I've had success with other sprint mvnos using the sprint gateways so definitely worth a shot
14:39 collum rhamby++
14:39 miker whee... Dyrcona / csharp, I'm about to push a change that 1) allows using a cpu per class (in different shells) 2) uses about 1/2 the RAM total 3) runs about 30% faster for the dictionary generation. then, you run the output of each and it upserts into the real table
14:52 sandbergja joined #evergreen
15:11 miker csharp / Dyrcona: I've pushed a commit to the same branch, with updated upgrade script instructions.  tl;dr: just value (no prefix) in the table dumps and name them for the class, run against each separately as they create class-specific unlogged files and then stitch the content into the main table with INSERT ON CONFLICT UPDATE
15:13 miker the RAM reduction speedup come from a simpler data structure per class -- hashes are heavy and slow -- and then let PG put it back together
15:13 terranm joined #evergreen
15:14 miker Dyrcona: I'll look at the rebase issues ASAP
15:14 Dyrcona miker++
15:16 miker (which should be this afternoon, but ... I suspect are committer-resolvable? but if you have a commit with the makefile changes handy, I wouldn't look /that/ gift horse in the mouth)
15:16 Dyrcona No, I don't have a commit handy.
15:20 miker kk
15:30 miker Dyrcona: rebased, CPAN modules updated, and pushed to https://git.evergreen-ils.org/?p=working/Eve​rgreen.git;a=shortlog;h=refs/heads/user/mike​r/lp-1893997-did-you-mean-rebase-2021-03-11
15:34 Dyrcona OK. I'll have a look in a bit. I'm trying the new steps, now.
15:41 mantis1 left #evergreen
15:52 terranm joined #evergreen
15:58 berick grabbing 1255
16:03 pinesol Showing latest 5 of 9 commits to Evergreen...
16:03 pinesol [evergreen|Chris Sharp] LP#1863252 - Use APT for Perl dependencies where possible. - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=bb22b04>
16:03 pinesol [evergreen|Jason Etheridge] lp1863252 make Geo::Coder::Free optional - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=7ca9a2b>
16:03 pinesol [evergreen|Jason Etheridge] lp1863252 fix Get Coordinates button in org admin - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=2ca0ae3>
16:03 pinesol [evergreen|Terran McCanna] LP1863252 Geosort - add dropped END tag - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=b991f5b>
16:03 pinesol [evergreen|Bill Erickson] LP1863252 Stamping DB Upgrade: Geosort - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=c992330>
16:03 gmcharlt berick++
16:09 Bmagic_ Has anyone put cloudflare proxy in front of a production instance of Evergreen? I did it once but immediately reverted because... problems. But that was in XUL days. What about now? Just wondering
16:17 terranm berick++
16:32 JBoyer Hi, a quick plea for time: I'm working on LP 1918470 because adding the html filter to mkurl has broken the place holds page (both the Continue link after success and the Cancel link / button)
16:32 pinesol Launchpad bug 1918470 in Evergreen "Internal Server Error After Placing a Hold" [Medium,Confirmed] https://launchpad.net/bugs/1918470
16:32 JBoyer WIP branch is at working/collab/jboyer/lp1918470_mkurl_fix
16:33 JBoyer I'm working on it but I figure it's better to raise a flag when *someone* might still be around today. :) (though looking at the clock, ugh.)
16:34 Dyrcona JBoyer: I can look at it tomorrow.
16:34 JBoyer Dyrcona++
16:39 jihpringle joined #evergreen
16:50 jvwoolf left #evergreen
17:00 Dyrcona Well, I'll have to finish up tomorrow. The identifier and keyword files are still running through the sideloader script.
17:00 Dyrcona Cheers, everyone!
17:31 mmorgan left #evergreen
17:33 sandbergja joined #evergreen
18:00 pinesol News from qatests: Failed Create Evergreen Database <http://testing.evergreen-ils.org/~live//arch​ive/2021-03/2021-03-11_16:00:05/test.41.html>
18:10 miker I'm pushing a fix for that -^
18:16 pinesol [evergreen|Mike Rylander] Minor mis-merge for geosort, missing comma in INSERT VALUES list addition. - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=b3e6738>
20:52 malexander joined #evergreen
21:08 jvwoolf1 joined #evergreen
22:45 sandbergja joined #evergreen
22:46 sandbergja joined #evergreen
23:26 sandbergja joined #evergreen

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat