Evergreen ILS Website

IRC log for #evergreen, 2019-09-12

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
01:55 StomproJ joined #evergreen
03:34 stephengwills joined #evergreen
06:30 agoben joined #evergreen
06:58 stephengwills joined #evergreen
07:00 collum joined #evergreen
07:12 rjackson_isl joined #evergreen
07:27 Dyrcona joined #evergreen
07:29 Dyrcona Hm... Restarted the load balancers and with "nothing" going on, ldirectord memory use is trending upwards again....
07:44 Dyrcona Yeah, this is pathological.  I wonder what's so special about the nginx proxy on the brick heads that it causes this behavior? I'm assuming that's the cause because we didn't have trouble with crashes/memory until after setting up the proxy.
07:52 Dyrcona "Nothing" going on, i.e. < 200 total connections, and ldirectord memory use is climbing by 20K per minute or so.
07:52 Dyrcona Nope. Much faster than that.
07:56 rfrasur joined #evergreen
08:04 JBoyer what kind of timeouts do you have in your nginx config? Maybe it's holding things open for too long?
08:09 bos20k joined #evergreen
08:11 JBoyer (seems unlikely since I know you've been looking into this a bit, but it's the thing that makes the most sense.)
08:19 _bott_ joined #evergreen
08:23 csharp Dyrcona: in case it helps, this is our nginx osrf config: https://pastebin.com/ggkhSMFj
08:23 csharp I do remember needing to adjust the timeout thresholds
08:23 Dyrcona Right now, I'm dealing with ldirectord working for a little bit, then showing no connections.
08:23 Dyrcona Thanks, I'll have a look when people can actually get in.
08:24 Dyrcona I think I'm switching to haproxy on the load balancer and getting rid of ldirectord and nginx in the not too distant future.
08:25 Dyrcona Since I can't seem to get any answers in #clusterlabs, should lidrectord only be running on 1 load balancer? The problems always seem to start a few minutes after I kill one.
08:27 Dyrcona Memory usages is still ballistic....
08:28 csharp <REDACTED>
08:28 csharp *sigh*
08:29 Dyrcona :)
08:30 Dyrcona Memory usage has gone up 4MB since I typed my "ballistic" line, and probably gone up more while typing this.
08:30 Dyrcona So, roughly 2MB/minute.
08:31 Dyrcona csharp: I'm going to try your proxy timeouts.  It can't hurt any more than everything else I've been doing.... :)
08:31 csharp yeah, I don't remember details atm but I know that solved a serious problem we were having at the time
08:35 Dyrcona Well, I've made the changes and reloaded nginx. We'll see how things go.
08:35 mmorgan joined #evergreen
08:36 Dyrcona websocket--- # Seriously: re-implementing TCP over HTTP, which is itself implemented on TCP, is a sign you're doing something very wrong.
08:36 Dyrcona And, that's not leveled at Evergreen specifically, that's a general commentary on the state of web development.
08:41 Dyrcona RSZ of ldirectord continues to creep upward on both load balancers.  Still don't know if only 1 should be running, but Evergreen is accessible, now, so I'm not touching it.
09:00 JBoyer Dyrcona, when you say "both," do you mean you have two ldirector load balancers up and running simultaneously for the same backend servers?
09:01 JBoyer I was under the impression you should only have 1 running and 1 ready to take over if there's an issue.
09:01 Dyrcona Yes, and it seems to be working.
09:01 JBoyer (i.e. up but not actually doing anything)
09:01 Dyrcona So was I, but when I rebooted them, they both started up. I killed the one with the least connections, and after a minute or two, no one could connect to Evergreen, and the one showed a handful of connections.
09:02 Dyrcona I have disabled ldirectord in systemd on both load balancers.
09:02 Dyrcona I let ha/heartbeat manage it.
09:03 Dyrcona Since it is working, now, and stopping one seems to cause us problems today, I'm leaving them alone for now.
09:03 JBoyer That's perfectly reasonable.
09:03 Dyrcona Now, I'm trying to find out more about this memory consumption. DDG is not proving very helpful as I keep find things from 2006 or so.
09:04 JBoyer I've never looked into using ha or heartbeat; I've only ever had a single ldirector running on its own, which while not entirely of the spirit of things, does work well.
09:06 Dyrcona Well, I think it is supposed to work that way: ldirectord runs on 1 load balancer and heart beat, or pacemaker or whatever they call it this minute, will start the other one if the first one dies.
09:06 Dyrcona How this is working with IP addressing I have no clue, but it seems to be working, and I'll settle for "wrong" if correct blows up in my face.
09:07 tlittle joined #evergreen
09:10 nfBurton joined #evergreen
09:28 Dyrcona Fun thing, too, because ipvs goes through the kernel, the problems could be kernel bugs and not just issues with ldirectord itself.
09:32 Dyrcona Memory use may have slowed. After 10 minutes it has gone up only 14MB.
09:40 mdriscoll joined #evergreen
09:42 yboston joined #evergreen
09:44 aabbee joined #evergreen
09:46 rfrasur joined #evergreen
10:52 Dyrcona csharp++ # For sharing the proxy config.
11:01 pinesol News from qatests: Testing Success <http://testing.evergreen-ils.org/~live>
11:18 csharp @praise Testing Success
11:18 * pinesol Testing Success LOVES the RESISTANCE!
11:20 jamesrf joined #evergreen
11:21 Dyrcona heh
11:21 Dyrcona @who loves the Resistance?
11:21 pinesol ericar loves the Resistance.
11:22 ericar wut
11:22 Dyrcona @who is a Spy?
11:22 pinesol dbs is a Spy.
11:22 Dyrcona I knew it!
11:24 dbs So true
11:24 dbs I've been unmasked
11:25 * Dyrcona is waiting on an action trigger run to finish to see if adding filters helps with something not related to ldirectord. :)
11:25 phasefx Resistance will always have to be an EG conference thing for me, my local gaming friends don't like it :D
11:27 Dyrcona That's too bad, but at least you have local gaming friends. I'm limited to mostly Scrabble, Yahtzee, and Dutch Blitz with the family.
11:28 phasefx kind of local.. 40 minute drive once or twice a month for that group.  1.5 hour drive once a week for D&D
11:28 Dyrcona D&D is worth it!
11:28 phasefx times for going in one direction
11:28 phasefx oh yes
11:28 * Dyrcona should find a group.
11:28 * JBoyer feels like a wet blanket re: games. So many bad experiences like Monopoly and so on that I tend to stick to the electronic games.
11:28 Dyrcona Monopoly is meant to be a bad experience.
11:29 dbs my daughter just joined a D&D group at her high school. i'm so proud
11:29 phasefx find a local adventurer's league (which sucks now, rules-wise), find a group there, and then poach them from AL into your own non-AL game :)
11:29 JBoyer It is, but when the game takes 4+ hours because of house rules, I hate it for reasons other than the intended purpose.
11:29 Dyrcona dbs: Cool.
11:30 Dyrcona @band add House Rules Monopoly
11:30 pinesol Dyrcona: Band 'House Rules Monopoly' added to list
11:31 Dyrcona The trouble with making configuration changes on low activity days is you can't tell if the changes really made an improvement.
11:32 Dyrcona Or if the numbers are just low because Thursday is usually less busy than Tuesday and Wednesday.
11:33 Christineb joined #evergreen
11:37 rfrasur joined #evergreen
11:40 nfBurton joined #evergreen
11:46 csharp my 16-year-old son is also in a D&D group
11:46 csharp @dnd
11:46 pinesol csharp: strength:11 dexterity:11 constitution:17 intelligence:13 wisdom:15 charisma:10
11:46 csharp @librarian
11:46 pinesol csharp: Management:16, Cataloging:10, Acquisitions:14, Reference:10, Circulation:17, Systems:8, Research:9, Custodial:14
11:51 * phasefx offers to run a game during hackaway
11:51 yboston joined #evergreen
11:52 berick phasefx: https://i.imgur.com/CJNovec.jpg
11:53 phasefx it's true!
11:54 JBoyer berick++
11:55 Dyrcona :)
12:07 jihpringle joined #evergreen
12:12 Dyrcona Well, that's nice: Can't locate object method "content" via package "OpenSRF::DomainObject::oilsMethodException" at /openils/bin/action_trigger_runner.pl line 174.
12:34 collum_ joined #evergreen
12:40 collum joined #evergreen
12:49 rfrasur joined #evergreen
13:00 khuckins joined #evergreen
13:07 cschroth joined #evergreen
13:08 HomerPublic joined #evergreen
13:14 yboston joined #evergreen
13:29 rfrasur joined #evergreen
13:33 JBoyer I know I've seen this before but I've forgotten everything about it apparently. I'm getting a JS error when loading the reporter, complaining that it can't insert the list of OUs into oils_rpt_top_folder_lib_picker.
13:34 JBoyer And in the logs there's an xml parser error looking at oils_rpt_editor.xhtml, preventing that page from being included in any other.
13:48 yboston joined #evergreen
14:04 yboston joined #evergreen
14:21 dbs JBoyer: are the two copies of the fm_IDL.xml (in /openils/conf and /openils/var/web/reports) in sync (assuming you're unilingual)?
14:33 JBoyer dbs, they are indeed.
14:39 dbs Well, that, and autoconf, is about the extent of my troubleshooting suggestions :/
14:50 JBoyer dbs++
15:05 jeffdavis gmcharlt: for bug 1817645 you had a couple of errata that should be fixed - should I open a separate bug for those now that the original is Fix Released?
15:05 pinesol Launchpad bug 1817645 in Evergreen "Configurable patron auth and retrieval" [Wishlist,Fix released] https://launchpad.net/bugs/1817645
15:06 gmcharlt jeffdavis: yes pelase
15:06 jeffdavis ok, will do
15:14 yboston joined #evergreen
15:21 frank joined #evergreen
15:23 frank Hi all, I have a question, I already had closed dates in EG by staff client, but I am trying to see them in web client and the window is empty, I mean, it does not display the closed date registerede previously by staff client, Do I have to save it againt by web client?
15:24 mmorgan frank: By default, the web client only shows closed dates in the future. You can change the date filter in the upper right corner of the screen and refresh
15:25 mmorgan1 joined #evergreen
15:26 khuckins joined #evergreen
15:56 csharp JBoyer: assuming it's behind a proxy, is the originating IP being passed through? (bouncing back the suggestion you had for me back in April with a similar issue :-) )
16:08 JBoyer csharp, looks that way, yep. (RemoteIP* settings in eg_vhost.conf are un-commented.)
16:09 csharp JBoyer++
16:09 JBoyer The worst part is that I've dealt with this in the last 3-4 months and forget what I did (or if the answer was "blow it all away and let /dev/null sort it out")
16:09 csharp @praise /dev/null
16:09 * pinesol You don't want to get mixed up with someone like /dev/null. /dev/null is a loner, Dottie. A rebel.
16:09 mmorgan joined #evergreen
16:14 frank joined #evergreen
16:15 frank mmorgan:  tks for responding, Do you know where does the closed dates are saved?
16:16 mmorgan frank: They are saved in the database table actor.org_unit_closed
16:54 frank mmorgan: thanks
17:00 mdriscoll left #evergreen
17:09 mmorgan left #evergreen
18:07 yboston joined #evergreen
18:29 khuckins joined #evergreen
19:48 cmalm joined #evergreen
20:08 ejk__ joined #evergreen
21:12 stephengwills joined #evergreen
23:01 pinesol News from qatests: Testing Success <http://testing.evergreen-ils.org/~live>

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat