Evergreen ILS Website

IRC log for #evergreen, 2019-04-05

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
02:14 sandbergja joined #evergreen
03:22 yar joined #evergreen
05:01 pinesol News from qatests: Failed Running pgTAP tests <http://testing.evergreen-ils.org/~li​ve/test.42.html#2019-04-05T04:57:49,268509913-0400 -0>
07:04 rjackson_isl joined #evergreen
08:17 bos20k joined #evergreen
08:29 Dyrcona joined #evergreen
08:37 mmorgan joined #evergreen
09:01 aabbee joined #evergreen
09:36 bos20k Hello. If anyone is having issues with SIP checkins being slow in recent versions of Evergreen, take a look at OpenSRF bug 1823338 https://bugs.launchpad.net/opensrf/+bug/1823338   Also, Evergreen bug 1803768 may be of interest since there is a workaround re copy limits but that workaround has a tradeoff.
09:36 pinesol Launchpad bug 1823338 in OpenSRF "SIP Checkin Speed vs Debug Logging" [Undecided,New]
09:36 pinesol Launchpad bug 1803768 in Evergreen "Checkins for age protected items on Bib Records with many holds can take a long time" [Medium,New] https://launchpad.net/bugs/1803768
09:37 Dyrcona Checkin is just plain slow.
09:37 Dyrcona It does too much, in my opinion, and staff are always asking for it to do more.
09:38 bos20k We found a debug logging statement in OpenSRF was making it unnecessarily slow.
09:39 bos20k Slow enough to make an automated item sorter almost useless.
09:39 Dyrcona Sure, there are things like that, but even without that, it's still doing too much. :)
09:40 bos20k Hehe, yeah, I agree feature creep is a major issue.
09:40 Dyrcona Yeah. I can see how performance would be an issue for an automated, well, anything. :)
09:42 Dyrcona bos20k: Couple of comments on your patch, intended to be constructive.
09:42 Dyrcona 1. It's best if the submitted patch is against the master branch, not a release branch, but in this particular case, it will likely still work on master.
09:43 Dyrcona 2. Do you have a ssh key on the git server? Most of us would rather look at git branches than patches.
09:44 bos20k Dyrcona: For 1, yeah, I know.  I should have redone the patch.
09:44 Dyrcona That said, it looks like a good change to me. I have a hunch that statement was not meant to be left in the code.
09:44 Dyrcona bos20k++
09:45 bos20k Dyrcona: For 2, yes, I think I do.  I assume to do that I need to set up a pull request?  Is there a document on doing that somewhere?
09:45 Dyrcona You just push your branch tot he working repository and add a comment on the Lp bug with the branch name and/or url. Add the pullrequest tag to the bug.
09:46 Dyrcona When/if we change to gitlab or to github, then we'll have actual pull requests.
09:46 bos20k Ok, I figured it was probably something like that.  Never had a reason to do it in the past.
09:47 aabbee https://wiki.evergreen-ils.org/doku.php?id=dev:git has a "quick start", if it helps
09:47 Dyrcona Yeah, I was about to post that. aabbee++
09:47 sandbergja joined #evergreen
09:48 Dyrcona bos20k: You have a key on the git server, with your first inital and last name as the user.
09:49 Dyrcona So, feel free to make a branch. It's good practice.
09:49 bos20k Thanks, Dyrcona++
09:50 mmorgan bos20k++
09:51 bos20k My gigantic question is are there other logging statements that are slowing things down elsewhere?  It isn't the call to the logger in this case, it is pulling the debug info together.
09:51 Dyrcona I'd also be inclined to just delete the line rather than comment it out.
09:52 Dyrcona Yeah, I'm sure there are lots of logging statements that slow things down. Have you ever really looked through the logs?
09:52 bos20k My thought is should be be pulling in the debug level from the config and then putting ifs around logging statements for the higher logging levels to avoid this type of issue.
09:53 Dyrcona I think logging could use a revamp. I find many of the log messages to be less than useful, and sometimes wish some things were logged that are not logged at the default level.
09:53 Dyrcona Well, the opensrf.xml config controls what log levels actually get printed so there's no need for if statements.
09:53 Dyrcona I don't think listing all of the methods in the log file is useful, at any logging level.
09:54 Dyrcona I do see your concern about putting the message together, even if it isn't printed, in this case.
09:55 bos20k Yes, but in this case it is pulling the info together that is slow, not actually logging it.  Even though we aren't running with debug loggin enabled it still slows us down.
09:55 Dyrcona Yeah. I think this particular line should just be dropped from the code.
09:56 Dyrcona Other places, if statements might be useful.
09:56 Dyrcona I'd rather not clutter the code up with a lot of "ifs", though.
09:56 Dyrcona It's kind of like #ifdef in C, when used like this.
09:57 bos20k If it results in significant speed improvements it might be worth it?
09:57 Dyrcona Well, I think each line should be considered on its own merits. I'm sure a lot of them can just be deleted with no harm done.
10:00 bos20k I had a crazy idea that I would analyze the current logging statements in OpenSRF and Evergreen to see if it would be possible and then I would write a program that would add ifs to all the Perl logging statements above a certain level (error or maybe info).  Combine that with having the logging module export the global logging level (something for the new ifs to check against).
10:01 Dyrcona That idea isn't so crazy: sed, awk, and perl were practically invented for that and similar purposes.
10:02 bos20k Automate the code changes.  The problem is to be effective developers would need to use the new if setup if this crazy change was merged.
10:02 Dyrcona And, for exporting the logging level, that would possibly be more useful than using SettingsClient and picking out the appropriate configuration.
10:03 Dyrcona Yes, well, it could be made a mandatory thing, and patches that don't do it could be rejected.
10:03 Dyrcona I want to reiterate, that I think it would be more beneficial to actually review the logging setup and eliminate any useless lines.
10:04 Dyrcona bos20k: I think you're heading in a good direction, don't get me wrong.
10:04 bos20k We have had many reports of slowness since our upgrade.  The SIP checkin slowness was a critical issue because it made a very expensive machine less useful.  But, maybe something like this could make significant performance improvements with no real downsides.
10:05 Dyrcona Anything that improves performance is a bonus. I'd also like to improve the maintainability of the code.
10:05 bos20k Dyrcona: totally agree, don't log stuff that isn't needed.
10:05 bos20k I think I'm going to start working on this and see where it takes me.
10:05 Dyrcona That sounds like a good idea to me. If you want any ideas/help, let me know.
10:06 bos20k Will do, thanks!
10:07 Dyrcona You're welcome and thank you!
10:11 sandbergja joined #evergreen
10:12 phasefx re: logging.. we could bundle any data gathering into a function to pass to the logger, and let the logger decide whether to invoke it or not based on the configured log level
10:12 phasefx if passed a string, log the string.. if passed a function, log the return value of the function
10:12 phasefx something like that
10:12 bos20k Interesting
10:13 bos20k How would that look for this: $log->debug("Available methods\n\t".join("\n\t", keys %{ $_METHODS[$proto] }), INTERNAL);
10:13 phasefx my perl is rusty, I forget how passing sub references work, if at all
10:14 bos20k Pulling that info together for the logger takes around 100ms.
10:15 miker phasefx: decompiling a function is ... sorta possible? ... with a debugging module, but it's not cheap
10:15 bos20k Pseudo code is fine.  My thought it to put if ($loglevel >= DEBUG) at the front of that line.
10:15 Dyrcona bos20k: To quote your bug report, "being called at least twice per checkin." You'll find a lot of that. I recall running checkins, checkouts, and renewals through the Perl debugger a few years ago, and saw many things happening repeatedly.
10:15 miker passing a sub ref would just get you "CODE" and an address, IIRC
10:15 phasefx in Javascript, functions are first-class objects :-(
10:16 Dyrcona bos20k: This being perl, you can put the if () on the end of the line if it's a one-liner.
10:16 Dyrcona That avoids having to wrap the lines inside { and }.
10:16 bos20k Dyrcona: good idea
10:17 Dyrcona phasefx has a good idea. passing subs is straightforward in Perl.
10:17 phasefx but has to be a named sub?
10:17 bos20k if ($loglevel >= DEBUG) at the end of the line then :)  ($loglevel or LOGLEVEL or whatever it ends up being would be exported from the logger module)
10:18 phasefx and you pass the reference
10:18 Dyrcona No. It doesn't have to be named.
10:18 phasefx sweet
10:18 Dyrcona $log->debug("Available methods\n\t".join("\n\t", sub {keys %{ $_METHODS[$proto] })}, INTERNAL);
10:18 Dyrcona That could work.
10:19 Dyrcona It could also be implemented to take a bare code block: $log->debug("Available methods\n\t".join("\n\t", {keys %{ $_METHODS[$proto] })}, INTERNAL);
10:19 miker fwiw, I think that line can just go away
10:19 bos20k Very interesting.  Not as easy to automate the code changes though...
10:19 Dyrcona or either a bare block or a sub reference. This is Perl, TIMTOWTDI!
10:19 phasefx $log->debug( sub { return "Available methods\n\t".join("\n\t",keys %etc. } );
10:19 miker it's probably from the first commit of opensrf code, for checking that methods we expect are actually getting registered at the right api level
10:20 miker narrator: they were all registered
10:20 Dyrcona Yeah, I guess I put the sub and block in the wrong place.
10:20 Dyrcona But, you get the idea.
10:21 Dyrcona miker and I agree on something! :P
10:21 miker Dyrcona: hey, it's not /that/ uncommon ;)
10:21 Dyrcona mker++ # For confirming my suspicion about that particular line.
10:22 * miker suspects we could probably just sed out all lines that contain INTERNAL in opensrf... anyone needing that verbosity can probably just add targetted INFO level calls for testing...
10:22 Dyrcona Actually, the log function could be written to take a plain string, a sub, a bare code block, well, just about anything.
10:23 Dyrcona Setting the log level to INTERNAL is a good way to fill your hard drive. :)
10:23 phasefx let it do Data::Dumper on anything not a string or code :)
10:24 Dyrcona phasefx: That's a possiblity.
10:24 phasefx except the output often gets truncated
10:25 Dyrcona Truncated output is better than ARRAY(somehashvalue)
10:25 * phasefx tends to data dump to a file when needed
10:25 Dyrcona Or whatever it looks like.
10:25 phasefx Dyrcona: true that
10:26 phasefx Dyrcona: I imagine data::dumper is relatively expensive though, so maybe we shouldn't do that, or should only do it for certain log levels
10:27 Dyrcona phasefx: Yeah, you are most likely correct.
10:47 berick Bmagic: in your copious free time, i'd love to know if the Hatch installer linked from bug 1817932 allows you to print to your Dymos
10:47 pinesol Launchpad bug 1817932 in Evergreen "Update Hatch Java / Javafx versions" [Undecided,Confirmed] https://launchpad.net/bugs/1817932
10:48 Bmagic TY!! I'm excited to try it (although I no longer have the printer physically) I do have the drivers on my Windows box though
11:00 gsams berick: I'm about to give it a test run.
11:04 berick gsams++
11:22 sandbergja joined #evergreen
11:23 sandbergja berick: I'm trying to wrap my head around the flesher method in the Angular 7 GridColumn class
11:23 sandbergja Would you be able to share an example for working with that?
11:25 berick sandbergja: i copied that over directly from angjs.  i've never actually used it.  FWIW, I'm on the fence about whether it should exist...
11:26 berick it's practically always better to give the grid the data it needs instead of asking it to fetch data in real time, which often leads to newtork call counts explosion
11:26 berick but the flesher could be used data that exists locally but for whatever reason is not in the data provided directly to the grid
11:27 Christineb joined #evergreen
11:27 sandbergja that makes sense
11:32 berick sandbergja: i didn't really answer your question..  i could put together a sandbox example if you want to use it.  clearly it has not been a priority for me.
11:33 sandbergja No worries -- now that I think about it, the thing that I need to do can totally be accomplished within my grid's getRows method
11:33 sandbergja no flesher necessary
11:35 berick cool
11:35 sandbergja So thanks for talking it out with me!  I'd been really hung up on thinking about the flesher, when there was a better way. :-)
11:35 sandbergja berick++
11:51 gsams berick: Okay, so we get printing, but it won't print correctly for us.
11:52 gsams It tries to print the spine and pocket label no matter how we set it up and we only need the spine.
11:53 gsams Also, don't test print with it.  It literally prints one character on each label from that, which was amusing.
11:53 Bmagic It's printing though!
11:53 gsams It is!
11:53 Bmagic that's great
11:53 gsams I'm very excited about that factor for sure
11:54 Bmagic and presumabely it's printing with javafx
11:54 berick gsams: i wonder if you should test printing "with dialog" and confirm java is using the right paper size
11:54 berick Bmagic: yes, using javafx
11:55 berick gsams: the spine label issue.. that sounds like a staff client problem
11:55 Bmagic mana from heaven. Let there be Dymo printing with javafx baby!
12:00 gsams berick: It is probably a staff client problem, but it was worth checking the label print size as it was technically incorrect for us.
12:01 gsams The label template ignores any changes I make to it on printing.
12:01 gsams So it's trying to print both the spine label and the pocket label.
12:02 gsams on a spine label.
12:03 gsams berick++
12:04 * berick sees a few spine label printing bugs open
12:04 berick i wonder if any of those are related
12:05 jihpringle joined #evergreen
12:17 berick sandbergja: FYI some of the items added to bug 1823367 may be of interest to your booking development
12:17 pinesol Launchpad bug 1823367 in Evergreen "Anguar staff catalog post-3.3 omnibus" [Wishlist,In progress] https://launchpad.net/bugs/1823367 - Assigned to Bill Erickson (berick)
12:17 sandbergja berick++ # I will take a look
12:32 * berick deploys omnibus to evgdemo.kcls.org
12:37 yboston joined #evergreen
13:08 jeffdavis Trying to get to the bookings UI in the web client, I am suddenly getting a 404 with "The requested URL /eg/eg/booking/reservation was not found on this server." in the iframe.
13:14 jeffdavis Possibly related to upgrading Apache to 2.4.18-2ubuntu3.10 on Ubuntu 16.04. The EG install itself hasn't changed.
13:18 jeff Was that the recent Apache release that changed how URLs are normalized?
13:20 jeff looks like yes.
13:24 jeffdavis jeff: do you have more info about that change?
13:24 jeff http://changelogs.ubuntu.com/changelogs/pool/mai​n/a/apache2/apache2_2.4.18-2ubuntu3.10/changelog has some bullet points, such as:
13:24 jeff - debian/patches/CVE-2019-0220-2.patch: fix r->parsed_uri.path safety
13:24 jeff in server/request.c, server/util.c.
13:24 pinesol ** <A HREF="https://cve.mitre.org/about/faqs.h​tml#reserved_signify_in_cve_entry">RESERVED</A> ** This candidate has been reserved by an organization or individual that will use it when announcing a new security problem.  When the candidate has been publicized, the details for this candidate will be provided. (http://cve.mitre.org/cgi-bin/c​vename.cgi?name=CVE-2019-0220)
13:25 jeff I can't guarantee that's it, but if you suspect the Apache update is what broke it, those changes seem worth looking at first.
13:26 jeffdavis ah, the "multiple consecutive slashes" thing
13:26 jeff Are you in any position to test with the previous Apache and then upgrade to this version? Be a shame to chase this down if it ends up not being Apache... I'd feel bad for encouraging you down this path, then. :-)
13:35 jeffdavis jeff: yep, booking works again after downgrading Apache with `aptitude install apache2=2.4.18-2ubuntu3`
13:38 jeff the iframe source on a system here shows a double slash: it contains /eg//eg/booking/reservation
13:38 * jeff looks to see why
13:40 jeffdavis (...and re-upgrading to 2.4.18-2ubuntu3.10 re-breaks things on the test system)
13:41 jeff Open-ILS/web/js/ui/default/staff/booking/app.js has: var booking_path = '/eg//eg/' + ...
13:42 jeff (in EmbedBookingCtl)
13:42 jeffdavis well, that would do it. :(
13:45 Bmagic Anyone going to get a copy of this game for the conference? https://www.kickstarter.com/projects/thedragonsto​mb/offensive-adult-party-game-by-the-dragons-tomb
13:51 jeffdavis Changing '/eg//eg/' to '/eg/' seems to work on both current and downgraded Apache. I'll report to Launchpad.
13:51 jeffdavis jeff++
13:53 jeff jeffdavis++
13:55 jeff I'm wondering how /eg//eg/ works in the downgraded apache...
13:55 jeff And starting to wonder if it has something to do with maybe-bugs we've seen with "/eg URLs break if a dir exists on the filesystem in that location". May have to poke at that again later.
13:56 jeff ...and I think we still have quirks with certain strings in hostnames.
13:56 jeff jeffdavis: Glad you mentioned the Apache upgrade being the likely culprit!
14:00 sandbergja jeffdavis++
14:00 sandbergja jeff+
14:00 sandbergja jeff++ # not enough plusses
14:00 sandbergja I just ran into that issue last night and got really confused.
14:01 jeff Only 240 unique values in actor.usr_address.address_type. I thought that number would be higher.
14:02 jeff (I've been wondering a bit about how people use the address type field.)
14:11 jeffdavis bug 1823387
14:11 pinesol Launchpad bug 1823387 in Evergreen "Apache update breaks booking UI" [Undecided,New] https://launchpad.net/bugs/1823387
14:32 Dyrcona jeffdavis++
14:53 yboston joined #evergreen
15:39 gsams Bmagic: So I figured out part of my printing issue for DYMO, but it's still not taking styling into account.
15:40 gsams it seems to print it in some sort of default style, like 9 size no bold no matter what I do.
15:42 gsams The fix for the earlier issue was that a template has to be assigned and applied for it to work, you can't just edit the current setup without naming it and saving it.
15:59 nfburton joined #evergreen
16:04 yboston joined #evergreen
16:06 berick gsams: using inline css?  like an inline <style> block?
16:06 gsams Yeah, that's what is used for that label template as far as I can tell.
16:07 berick k
16:13 berick gsams: may be worth checking what HTML Hatch is receiving.  you can see the full messages in the hatch log files
16:14 berick in home directory, in .evergreen directory
16:24 yboston joined #evergreen
16:49 yboston joined #evergreen
17:01 pinesol News from qatests: Failed Running pgTAP tests <http://testing.evergreen-ils.org/~li​ve/test.42.html#2019-04-05T16:58:13,393702099-0400 -0>
17:11 gsams berick: It looks like it's sending the correct information.
17:12 mmorgan left #evergreen
17:14 pastebot "gsams" at 64.57.241.14 pasted "Log of last print attempt" (8 lines) at http://paste.evergreen-ils.org/10842
17:18 gsams actually, now that I'm looking more carefully I think it may have been leaving out the style.
17:18 gsams will test further.
17:31 berick i'd also try copying the stock template out for safe keeping and trying a very simple template
17:32 berick just a bit of style and some hard-coded text
17:32 berick to rule out template issues
17:38 gsams I have done that, it ignores inline css entirely.
17:39 berick as in, it does not get delivered to Hatch?  or it's there, but Hatch ignores it?
17:40 gsams Hatch is getting the css, I'm not sure that it's ignoring it though.
17:40 berick also curious if you have tried element-level css, like <div style="font:..." />
17:41 berick i'm positive hatch java bits handle css
17:41 gsams That is what I tried most recently, instead of relying on the spine style.
17:41 gsams er class.
17:41 berick k
17:42 gsams I think it may be the DYMO, and maybe I'm missing something.
17:48 berick certainly worth trying one of the test templates against another printer or print-to-file
17:49 berick of course the dymo doesn't see css, just the final (headless) browser view
17:49 berick but it could be some limitation of font types / sizes in the dymo?
17:57 gsams_ joined #evergreen
17:58 abneiman_ joined #evergreen
17:59 book`_ joined #evergreen
18:00 pinesol` joined #evergreen
18:01 berick_ joined #evergreen
18:02 ningalls_ joined #evergreen
18:02 aabbee joined #evergreen
18:05 dbs joined #evergreen
21:05 sandbergja joined #evergreen

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat