Evergreen ILS Website

IRC log for #evergreen, 2019-09-17

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
02:39 dbs joined #evergreen
06:59 agoben joined #evergreen
07:13 rjackson_isl joined #evergreen
07:26 rfrasur joined #evergreen
07:26 Dyrcona joined #evergreen
08:46 mmorgan joined #evergreen
08:48 jvwoolf joined #evergreen
08:53 sandbergja joined #evergreen
09:18 tlittle joined #evergreen
09:19 csharp any major sites running PG 10+?  I'm upgrading our next-release test servers to Ubuntu 18.04 which comes with 10 and I'm trying out PG11
09:31 yboston joined #evergreen
09:49 Stompro joined #evergreen
10:14 * Dyrcona shakes his fist at slow websites and networks everywhere.
10:14 collum joined #evergreen
10:15 alynn26 Slow websites --
10:34 bshum csharp: Be curious what you find, we keep working on it, but not enough data to know :)
10:35 Dyrcona csharp: We've not looked at PG11, yet. I've meant to start, but other issues have come up.
10:35 Dyrcona As for Pg10, I don't think anyone has tried it in any serious way.
10:35 bshum That's what I think too
10:36 bshum Getting the community to standardize on PG9.6 was the latest official step
10:36 bshum For the next release
10:36 Dyrcona bshum and I have rune the tests, and JBoyer found a bug with Vandelay that tests don't cover.
10:36 bshum And we fixed that
10:36 bshum At least I think it went into master
10:36 Dyrcona Yeah, we did fix it.
10:36 Dyrcona It was committed.
10:37 Dyrcona csharp: God speed and good luck! :)
10:37 bshum Huzzah!
10:39 Bmagic A quick double check: when AT templates result in nothing in the "To:" address, will it attempt to pass it to the mailer? And result in an error at the Sendmail level?
10:42 Bmagic I don't see anything in my mail logs with to=<>
10:43 Dyrcona Bmagic: Yes. See bug 1837454
10:43 pinesol Launchpad bug 1837454 in Evergreen "SendEmail Reactor Will Try to Send Email with no valid To, Cc, or Bcc Headers" [Undecided,New] https://launchpad.net/bugs/1837454
10:44 Bmagic Dyrcona++
10:45 Bmagic I wonder if there is a repercussion when using a relay service like sendgrid? Would it debit the allowed "bounces"
10:46 Dyrcona Dunno.  We have exim set up to relay via Google.  That error in the bug description is from exim, so the mail never goes anywhere in our case.
10:47 Bmagic I guess the question is: would sendmail even attempt to deliver the envelope to the relay (if sendmail is used on the AT server)
10:49 Christineb joined #evergreen
11:00 Dyrcona Bmagic: I don't use sendmail, so I don't know, but I doubt it.
11:01 Dyrcona Bmagic: You could search your logs for something like "No recipient addresses found in header"
11:01 Bmagic looking
11:02 pinesol News from qatests: Testing Success <http://testing.evergreen-ils.org/~live>
11:04 Bmagic Nothing like that in the logs
11:05 Dyrcona You searched all of the logs, not just Evergreen logs?
11:05 Bmagic /var/log/mail.log*
11:06 Dyrcona There may be different logs for sendmail, but I haven't used it in over 17 years.
11:06 Bmagic :), no worries, thanks for thinking about it with me
11:06 Dyrcona That also varies by O/S and distro.
11:06 Dyrcona I really doubt that sendmail is relaying mail with no recipient, though.
11:07 Bmagic I am thinking that's the case based on the log "tea leaves"
11:09 berick Bmagic: i would expect Email::Send would crap out on the message before relaying.
11:09 Bmagic berick: yep, that's exactly what I was looking for
11:09 berick which would result in evergreen logs like "SendEmail Reactor: unable to send ..."
11:10 Bmagic SendEmail Reactor: unable to send email: $VAR1 = bless( {'prop' => {},'string' => 'Can\'t send data','errno' => 1,'type' => 'failure'}, 'Return::Value' );
11:10 Bmagic SendEmail Reactor: failed email template: To: From:
11:10 berick yeah
11:10 Bmagic yep, there's the smoking gun
11:12 Bmagic I was dreaming of altering the template to intentionally blank the To: address under some IF condition and wondering if that would be a bad thing. I think it would be "ok"
11:13 Dyrcona I find a few of those messages from today.
11:14 Dyrcona Well... The template shouldn't be generated if there's no recipient email address.
11:14 Dyrcona Hence, the chained validators bit.
11:15 Bmagic there should be a way to short circuit the whole template to make a completely blank result (no headers, no nothing) at the template generation level. Of course, a better way would be to add validators into Evergreen as you suggest
11:15 mmorgan Bmagic: Under what circumstances would you want to blank the To: address? What are you trying to accomplish?
11:16 Bmagic the library would like certain patron groups to be removed from reactor SendEmail, SendSMS
11:17 Dyrcona Our email setup is pretty crufty, and I don't think it even works on Perl versions higher than 5.22. There are a number of email bugs assigned to me that are so far going nowhere.
11:18 Dyrcona Bmagic: You can do that with a filter on the events that run those reactors, I think.  I'm not sure if filters work on events fired by a previous event, i.e AutorenewNotify.
11:18 mmorgan Bmagic: What about a custom a_t_filter to filter out those patron groups....
11:18 Bmagic opt-in comes to mind as well
11:18 mmorgan What Dyrcona said:)
11:19 Dyrcona Thing is, you might want the main event to trigger but not the notification event, and I've not looked at how filters interact with those.
11:20 Bmagic the filter sounds attractive
11:21 Dyrcona Well, all right, there is 1 email bug assigned to me, that I think I'm going to drop for now.  I also need to take a look at the 9 other bugs and get some testing scheduled, but other things have been a higher priority....
11:29 jeff I may pick that up, if it's the one I think it is.
11:31 Dyrcona jeff: https://bugs.launchpad.net/evergreen/+bug/1466502
11:31 pinesol Launchpad bug 1466502 in Evergreen "Replace Email::Send with Email::Sender" [Wishlist,Confirmed]
11:32 Dyrcona I was looking at this one, but dropped it some time ago: https://bugs.launchpad.net/evergreen/+bug/1801163
11:32 pinesol Launchpad bug 1801163 in Evergreen "SendEmail A/T reactor broken for recent version of Encode::MIME::Header" [High,Confirmed] - Assigned to Jeff Godin (jgodin)
11:32 Dyrcona jeff: I see you grabbed the latter.
11:33 * Dyrcona runs out to get some lunch.
11:49 Stompro Bmagic, we pre-process action_trigger pending runs with a sql script to do things like that, like removing certain permission_groups, and just mark those events as invalid.
11:49 Bmagic Stompro: interesting
11:50 Stompro Bmagic, it is also how we roll failed bill emails over to print bills, and limit sms & phone calls to one per day per patron.
11:51 Bmagic very cool
11:51 Bmagic got a branch on that?
11:52 Stompro No, but I can put it in a gist.
11:52 mmorgan Stompro: That is cool!
11:52 Bmagic That sounds like a worth-while feature for everyone!
11:53 Stompro It isn't as maintainable as the built in methods, and requires editing scripts on the server.
11:54 rfrasur joined #evergreen
11:54 Bmagic editing scripts? oh, well, in that case nevermind.... haha
11:54 mmorgan I'd still favor filters wherever possible to prevent certain events from being created in the first place.
11:56 jvwoolf joined #evergreen
11:57 Stompro mmorgan, filters cannot specify joins though, so no way to add in criteria like patron email exists as far as I could tell.
12:01 aabbee joined #evergreen
12:02 * mmorgan thought someone had shared a way to do that in a filter on irc at one point. Can't locate it though, and never got a chance to try it.
12:04 Dyrcona mmorgan Stompro: You can specify subqueries.
12:06 Dyrcona Pretty much all of our custom filters have an '-and' : [ {"-exists" : { "select" : ....
12:06 Stompro Dyrcona, could you share your custom filters please, I would love to see examples.
12:07 berick Bmagic: FYI, https://git.evergreen-ils.org/?p=worki​ng/Evergreen.git;a=shortlog;h=refs/hea​ds/user/berick/el-integrate-circa-3.4 -- wasn't really ready to start talking about it, but I'm hoping to start testing locally before too long
12:07 jihpringle joined #evergreen
12:08 Bmagic berick++ # is there more on this topic that I am not aware of?
12:09 berick Bmagic: no, so far it's just been a local project.
12:09 berick borrowing an idea or 2 from jeff's mobile/elastic catalog
12:09 Stompro Bmagic, here is an example of how we pre-process AT events.  https://gist.github.com/stompro/​45bc96af05c015769a95d7b5b8e99316
12:11 jeff berick++ bookmarking for later
12:11 Stompro Dyrcona, I would love to update the AT filter docs with info like this, the json sql syntax really makes my brain hurt though.
12:11 Bmagic berick: nice! Glad I didn't get too far into it myself!
12:11 Bmagic Stompro++
12:13 Dyrcona Stompro: Here's a simple example for our member who sends lost notices after 14 days: https://pastebin.com/CECvqLqx
12:14 mmorgan Dyrcona++
12:14 Dyrcona Limits to certain circ modifiers via the subquery.
12:14 Bmagic Stompro: and here is mine https://pastebin.com/ZmRJ7mHp
12:16 Dyrcona You could use that to limit to patrons with an email, or a SMS setting.
12:16 jvwoolf1 joined #evergreen
12:17 Stompro Dyrcona++  Bmagic++ thanks, I think I need to go back and amend some emails to the listserve now.  "New information (to me) has come to light"
12:19 Bmagic berick: wow man, you've been busy! This is great! Mind if I link it to the LP?
12:21 berick Bmagic: no, i don't mind
12:22 Dyrcona berick++
12:23 Bmagic berick++ # ferreal
12:23 sandbergja berick++
12:24 khaun joined #evergreen
12:24 Bmagic I'd say it was something close to telepathy that I posted that LP bug today only 7 days after this code started appearing from berick
12:24 berick Bmagic: oh, no, it was rebased 7 days ago
12:24 berick it's been in progress for years now
12:24 Bmagic well I guess I take that back then
12:25 berick but over last month i've been putting what I think are the finishing touches
12:25 Bmagic I have to admit: It's been on my mind on and off for years now.
12:25 berick at least enough to support the staff work flow we need
12:25 Bmagic and just recently got serious about it
12:28 Bmagic Now I have to re-plan my freed 6-18 month future. Thanks a lot berick!
12:30 berick heh, well, let me know if you have questions, etc.  eyes and testing obviously appreciated
12:31 * berick should probably post a tech doc to the repo
12:39 jvwoolf joined #evergreen
13:18 nfBurton joined #evergreen
13:21 khuckins joined #evergreen
13:28 * jeff eyes a 3.1 db where most of the vr_format values in config.coded_value_map are duplicated once
13:28 sandbergja joined #evergreen
13:29 jeff (all but " " Unspecified and "s" Blu-ray disc)
13:36 rfrasur joined #evergreen
14:04 bos20k joined #evergreen
14:06 jvwoolf joined #evergreen
14:29 khuckins joined #evergreen
14:54 jeffdavis How do you all handle Postgres upgrades? Do you just do pg_dump and restore to a new cluster, or use pg_upgrade, or...?
14:58 mmorgan1 joined #evergreen
15:00 Dyrcona jeffdavis: When I upgraded from Pg 9.2(?) to Pg 9.5 I also got new servers, so I did a dump and restore. I have done some test upgrades using pg_upgrade and that seems to work, too. I'm planning to use pg_upgrade to go from 9.5 to 9.6 (or maybe 10) before we upgrade to Evergreen 3.4 next spring.
15:01 Dyrcona Having replication makes it more interesting, though you can disable replication, upgrade the software on the replicant, to a pg init db, and start replication over from scratch if anything goes wrong.
15:03 gmcharlt jeffdavis: we use pg_upgradecluster most of the time
15:04 jihpringle joined #evergreen
15:16 bwillis joined #evergreen
15:20 bwillis I am trying to add some new Z39.50 targets in Evergreen.  I am using the The Z39.50 Target Directory compiled by IndexData (http://irspy.indexdata.com/) I have had success connecting to only one of several targets that I have tested.  I don't know if the directory information is incorrect or if the targets I selected require authorization.  Can
15:20 bwillis anyone recommend one or more targets that your system is currently using?
15:22 jeffdavis Dyrcona, gmcharlt: thanks!
15:29 bwillis53 joined #evergreen
15:33 bwillis3856 joined #evergreen
15:36 dbs @later tell bwillis it can be tricky to set up new Z39.50 targets, things like truncation and mapping to fields correctly make a big difference. As does the need for authentication (in some cases)
15:36 pinesol dbs: The operation succeeded.
15:37 bjwillis joined #evergreen
15:38 dbs bjwillis it can be tricky to set up new Z39.50 targets, things like truncation and mapping to fields correctly make a big difference. As does the need for authentication (in some cases)
15:38 Stompro @later tell bwillis bwillis++ that is a great project.  I have had so little luck adding Z39.50 targets that I cannot be any help.  We wanted to add a library across the river from us, they were willing, we were willing, but no one could tell us the correct settings and I had no luck figuring it out.
15:38 pinesol Stompro: The operation succeeded.
15:40 dbs bjwillis: https://pastebin.com/xa6hA4tJ is an example of one that I think we have working...
15:40 dbs (sorry for the database format but I'm pressed for time)
15:41 bjwillis dbs: thank you for that example!
15:42 dbs I've found it confusing myself, most recently: http://irc.evergreen-ils.org/​evergreen/2019-07-12#i_412193
15:43 bjwillis I should mention that I have had success adding a few targets, but had hoped to add a few more that might be more relevant to our libraries
15:44 dbs Ah, sounds like a good wiki-based community project. Or SQL maintained in a "contrib" git repository or something like that!
15:44 dbs It can be really frustrating when Z39.50 servers advertise UTF8 but actually give you MARC8 and the like.
15:46 Dyrcona There is this: https://wiki.evergreen-ils.org/doku.p​hp?id=evergreen-admin:sru_and_z39.50
15:47 Dyrcona But that's mostly about setting up your Z39.50 server.
15:48 bjwillis dbs: that IS confusing!
15:49 Dyrcona There's a dokuwiki page from 2009, but I hesitate to share the link because it looks out of date.
15:49 Dyrcona bshum and I were working on some instructions for setting up another Evergreen instance some years ago, but I can't find them, now.
15:51 bjwillis Can someone tell me if the "Format" attribute in the Z39.50 target configuration refers to the Bib-1 Structure Attributes?
15:54 jihpringle bjwillis: I couldn't figure out what Format is.  I think we have it set to 1 for most or all of our sources
15:54 phasefx I think it's for the fullness of the record
15:56 phasefx bjwillis: it is the Bib-1 Structure Attributes
15:56 phasefx raw, it gets sent with @attr 4=
15:59 jihpringle just as an fyi while we're talking about Z39.50 attributes - https://bugs.launchpad.net/evergreen/+bug/1834687
15:59 pinesol Launchpad bug 1834687 in Evergreen 3.3 "Z39.50 Servers Attributes page missing" [Medium,Confirmed]
15:59 * phasefx may be jumping the gun, 3 different format fields with z39.50  in the source code :)
16:01 phasefx the one associated with attributes is the structure thingy
16:01 jihpringle In the search attributes I've seen from Z39.50 sources they have Position, Structure, and Completeness fields that we don't have (or don't have labeled as such in Evergreen)
16:01 jeffdavis jihpringle: berick pointed us at part of the solution for that bug last week - it is already possible to filter the attributes grid by Z39.50 source via URL param. I have been trying to figure out how to add a link to the Z39.50 Sources UI that links to the appropriately-filtered page.
16:01 jeffdavis I should update the bug with that information.
16:02 jihpringle great, thanks jeffdavis
16:11 yboston joined #evergreen
16:34 jvwoolf joined #evergreen
16:42 cmalm joined #evergreen
16:48 CMalmgren joined #evergreen
16:49 CMalmgren left #evergreen
16:49 CMalmgren joined #evergreen
17:03 mmorgan1 left #evergreen
17:14 jvwoolf joined #evergreen
17:27 sandbergja_ joined #evergreen
18:04 cmalm joined #evergreen
18:18 khuckins joined #evergreen
21:36 sandbergja joined #evergreen
22:56 sandbergja joined #evergreen
23:03 pinesol News from qatests: Testing Success <http://testing.evergreen-ils.org/~live>
23:28 jamesrf joined #evergreen

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat