Evergreen ILS Website

IRC log for #evergreen, 2015-11-10

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
01:05 tsbere_ joined #evergreen
02:48 phil joined #evergreen
02:49 phillipjphry Hello!
03:06 phillipjphry I'm a library worker in BC setting up Evergreen for the heck of it. I'm on the last step in Evergreen/opensrf setup
03:07 phillipjphry when I run the settings-tester.pl script, it hangs on testing connections to the database, and I think something went funny with my perl database setup script, as it just hangs(and is STILL hanging)
03:21 phillipjphry Woop. Looks like I was just too hasty. I opened top and saw root had an init script running, and then some stuff popped up, one of them being the message about joining this channel for help.
03:21 phillipjphry so I guess I'll just let it run. Silly me.
03:33 phillipjphry Ok, completed, but threw an error, trying the settings tester again, just going to be patient this time
04:39 phillipjphry Sorry about all the late night spam.
05:05 phillipjphry AArrrrrggggghhhhh. I ran through everything, including the troubleshooting guide, and well, now apache won't start. I'm going to just wipe it and try again tomorrow night.
05:05 phillipjphry Thanks for the wall to yell at. I'll try not to step on anyone's toes.
05:12 phillipjphry fishyjoes.com is where I've posted my problems. If anyone feels like following up.
05:12 phillipjphry Thank you.
07:38 ericar joined #evergreen
07:45 rjackson_isl joined #evergreen
07:56 csharp phillipjphry: what instructions are you going by?
07:57 jboyer-isl joined #evergreen
08:03 graced joined #evergreen
08:13 mrpeters joined #evergreen
08:33 collum joined #evergreen
08:35 jeff phillipjphry: my first reaction is that you may be trying this on a system that does not have enough RAM to successfully complete the tasks. once things start swapping due to there being little free memory, things will get very slow. after that, you'll reach a point where the kernel Out of Memory killer will terminate processes (you'd see that happen in logs, likely also visible in the output of the dmesg command)
08:35 jeff phillipjphry: and once the OOM killer steps in, things are a mess.
08:36 jeff phillipjphry: it sounds like you're also on the correct path with regard to certain pre-requisites not being installed. missing pre-reqs will cause various Evergreen components to fail to start or fail to operate properly.
08:37 mmorgan joined #evergreen
08:46 Dyrcona joined #evergreen
08:49 Dyrcona Figures.... Kernel update.
08:52 jeff oh good. my index script didn't encounter any more unexpected exceptions last night.
08:52 jeff (i should have known that the assumption of "every bib will have a title" was completely invalid)
08:54 Dyrcona joined #evergreen
08:54 Dyrcona Bib without titles!? Inconceivable!
08:55 RoganH joined #evergreen
08:56 Shae joined #evergreen
08:57 jeff yeah.
08:57 jeff To be more accurate, it wasn't that I was making that explicit assumption, I was just failing to properly account for the possibility when I added some new code to process titles.
08:58 jeff I find myself wanting to write search tests.
08:59 Dyrcona I mean titles are kind of the whole point, no?
08:59 jeff Not hard pass/fail so much as "run this test suite to get an idea of how these known searches are performing with regard to these expected results"
08:59 Dyrcona Heh. "Bibs without titles!? Pointless!"
08:59 jeff "a search for FOO should turn up the following record or records within the first X results", that kind of thing.
09:00 jeff Dyrcona: "I don't know the title, but it was blue..."
09:00 Dyrcona jeff: Sorry, we weeded the blue books yesterday. They were depressing the staff.
09:01 Dyrcona jeff: Have a lovely, yellow book! It's cheerful!
09:01 * Dyrcona stops channeling Michael Palin.
09:01 jeff :-)
09:02 jeff I used crayola crayon colors as a naming scheme once upon a time.
09:02 * Dyrcona feels like a kid hopped on sugar.
09:03 Dyrcona When I stayed with a family in France, the daughter arranged the books on the shelf by size.
09:04 Dyrcona I don't dare touch the books on the shelf over the desk at home. Move one, and the rest come toppling down.
09:05 dbwells jeff: Having search tests like you are talking about would be a very valuable thing.  Maybe even necessary for regression purposes if we want to try to make bigger changes.
09:06 dbwells It's really easy to improve specific searches, much harder to improve them all together :)
09:08 jeff yeah. in experimenting with Elasticsearch over the past few weeks, we've been quite aware that pushing something down over here pushes a bunch of things up over here, etc.
09:09 jeff (a concept not unique to Elasticsearch, and one that I heard at least one other person reference during hack-a-way discussions)
09:09 jeff I've been trying to think what it reminds me of.
09:10 jeff Almost but not quite some amalgamation of the tale of the Little Dutch Boy and something from looney tunes.
09:19 * Dyrcona imagines Marvin the Martian with his finger over a hold in a dike.
09:19 Dyrcona gah...
09:20 Dyrcona s/hold/hole/
09:29 mmorgan1 joined #evergreen
09:30 mmorgan2 joined #evergreen
09:32 mmorgan3 joined #evergreen
09:34 ericar_ joined #evergreen
09:44 yboston joined #evergreen
10:07 gmcharlt Dyrcona: better watch out for the holds as well... you know they have a habit of multiplying!
10:13 Dyrcona gmcharlt++
10:15 rjackson_isl Happy 240th to the USMC (off to boot camp 38 years ago today...)
10:16 Dyrcona rjackson_isl: "Don't dial 911. Dial 0311." ;) I saw that on a bumper sticker this morning, in fact.
10:17 rjackson_isl :)
10:27 dbs @dial 0118 999 881 99 9119 7253
10:27 pinesol_green dbs: Sorry, that command is only available to Evergreen Premium™ Subscribers. Please upgrade your subscription ASAP!
10:31 jeff Hrm. I think I may expand upon the idea that I can index things to an intermediate format then batch insert them into Elasticsearch when I make a reindex-requiring change to my index mappings.
10:32 jeff So far, I've been doing a full re-parse of the MARC data, mostly because I've been making changes on both sides.
10:33 jeff (changes in how I'm parsing the MARC data, and changes to the Elasticsearch index mapping config)
10:33 jeff But in cases where I'm only changing the Elasticsearch side of the equation, I'll bet the intermediate bit could speed that up greatly.
10:34 jeff Actually, there's already existing work/examples of that kind of thing.
10:57 Stompro Does anyone know off the top of their heads what needs to be restarted after an AddedContent provider change?
10:58 Stompro The docs just say "Evergreen and Apache" but I would rather just restart the specific service.
10:58 jeff Stompro: what have you changed?
10:59 Stompro changed from openlibrary to content cafe.
10:59 Stompro In opensrf.xml
11:01 jeff did you also modify the SetEnv statements for content cafe in your apache config?
11:01 bshum I would imagine restarting memcache would be helpful too, to get rid of any cached openlibrary data in favor of contentcafe stuff?
11:02 bshum (which mean restarting everything else)
11:02 Stompro Nope, but we just use the cover art, none of the other features.
11:02 jeff off the top of my head, you'll want to restart the settings service, apache, and consider clearing out cached ac keys from memcached.
11:02 krvmga joined #evergreen
11:02 Stompro I don't want to invalidate logins, so I'll just let the current memcache entries expire.
11:02 jeff keep in mind if you completely restart memcached you will invalidate all existing login sessions, including staff clients and SIP clients, etc.
11:03 Stompro Hehe, great minds.
11:04 jeff i think settings and apache would be enough, but don't take that as authoritative. i'm not in a position time-wise to check my thinking.
11:04 Stompro jeff, thanks for the suggestions.
11:04 berick that sounds right.  and apache reload should suffice.
11:05 jeff keep in mind that memcached caching will also include negative caching of "not found" cover images, etc.
11:17 csharp dbs++ # "well THAT's easy to remember!"
11:19 jeff woo!
11:19 jeff es = Elasticsearch([config['elasticsearch']['url']])
11:19 jeff reindex_result = elasticsearch.helpers.reindex(es, 'records-v7', 'records-dev')
11:20 jeff pulls all records from records-v7 and inserts them into records-dev
11:20 jeff there are in-process plugins that can do it without the data having to pass from server to client and back, but this works well for my needs.
11:23 dbwells rjackson_isl++
11:24 rjackson_isl dbwells thanks!
11:26 jwoodard joined #evergreen
11:34 cja joined #evergreen
11:35 jeff aww. ten minutes in, i find out that i'm using the default client_max_body_size of 1M for nginx. :-)
11:38 Dyrcona jeff: How are you liking nginx?
11:38 * Dyrcona has been wanting to try it out, but hasn't made the time to do so.
11:41 * dbs uses nginx for a lot of other projects
11:41 jeff i like it reasonably well. been using it for various things starting with a frontend proxy for GroupWise WebAccess.
11:41 jeff also an old IIS 4 box.
11:42 jeff both of which are dead, thank goodness. :-)
11:42 jeff but it was what i grabbed when i needed a layer in front of Elasticsearch for auth.
11:42 csharp one of the problems we're seeing with credit card processing setup is that the vendors are requiring an individual's personal financial information - is that the norm?  We were expecting that an institutional account not tied to a person was available...
11:42 jeff csharp: which vendor?
11:42 csharp PayPal and Stripe, so far
11:43 jeff csharp: i don't remember having that issue with authorize.net
11:43 csharp hmm - I'll have to take a look
11:43 jeff for Stripe, I was pretty sure they took an EIN instead of an SSN.
11:43 csharp ah - maybe it's how they're registering
11:44 jeff for Square, I believe we needed an individual for verification but after that it wasn't like it was tied that that individual's bank account or anything.
11:46 jeff Stripe does appear to require identity verification for at least one representative of the organization.
11:46 csharp yeah, I just got a clarification - it'
11:47 csharp s for identity verification, I think
11:47 csharp the way it was coming to me sounded like it was requiring an individual's bank account, etc.
11:47 miker csharp: that's normal to set up a CC processing account
11:47 _bott_ joined #evergreen
11:48 csharp yeah, I can see that - I think I was misunderstanding what was being told to me
11:48 jeff stripe goes into a little more detail here: https://stripe.com/us/terms#v​erification-and-underwriting
11:48 miker csharp: gov't entities may have different requirements with CC processors, fwiw... do they know PINES is a state dept?
11:49 csharp yeah - it's not PINES - it's individual libraries...  We're playing a facilitating role here so far
11:50 jeff the default business types for Stripe don't include any good fits: Corporation; Individual/Sole Proprietorship; Non-profit; Partnership; LLC
11:50 csharp jeff: thanks for the link
11:51 miker csharp: ah, gotcha ... well, in GA, at least, that's still gov't, yeah?
11:51 csharp miker: yep
11:51 csharp jeff: we're also seeing that our libs don't really fit in PayPal's categories either
11:52 miker may take a call from PINES to find someone at the processor and convince them to be the account rep for any GA libraries you bring over... to smooth the way
11:52 berick csharp: i can confirm we are registered via KCLS the entity for payflowpro, not a personal account
11:52 csharp berick: thanks - good to know
11:54 csharp weird - I just hit some key combination that made my dual displays stack on top of each other :-/ (running Ubuntu 15.10)
11:55 berick csharp: that's it, you'll have to reinstall ubuntu
11:58 Dyrcona heh.
12:00 jeff hah. nice:
12:00 jeff 2015-11-10 11:39:29,862 INFO: Starting reindex...
12:00 jeff 2015-11-10 11:52:27,267 INFO: Reindex result: (264555, 0)
12:00 jeff real    12m57.612s
12:05 jeff so that's "i created a new index with some different mapping settings, copy the records from the old index to the new index", and 265k records were copied in ~13 minutes.
12:16 berick jeff: just curious, do you know the highest copy-to-record count in your sample set?
12:16 berick or average, even
12:21 bmills joined #evergreen
12:31 jeff average is 1.3 copies per record, but i'm sure that's dragged down a bit by things like located uris, which have no "copy" in this set.
12:36 Stompro kmlussier++ congrats on becoming a core committer. (I'm just catching up on the news)
12:41 phasefx kmlussier++
12:42 mmorgan kmlussier++
12:48 Shae joined #evergreen
12:52 berick jeff: i need to send you some of our 'time' and 'people' magazine records...
12:52 berick 'people' is almost at the 4k copies mark
12:52 jeff berick: i did realize the other day that i am indexing bib -1 :-)
12:52 berick ah, yeah, i bet that was fun
12:53 jihpringle joined #evergreen
12:54 jeff oh, or maybe i'm not.
13:00 jeff i was, and am no longer. unintentional bugfix? :-)
13:06 maryj joined #evergreen
13:25 krvmga who takes care of git.evergreen-ils.org?
13:26 tsbere krvmga: Depends on what you mean by "takes care of"
13:27 krvmga tsbere: the root of my question is i'd like to set something like that up for c/wmars; where we could look at what we're working on
13:27 tsbere ahhh
13:27 tsbere it isn't hard. MVLC has stuff set up too.
13:28 krvmga cool
13:28 tsbere krvmga: http://git.mvlcstaff.org/
13:29 krvmga tsbere++
13:30 tsbere krvmga: Generally it is a combination of gitolite and gitweb. The former for access control, the latter is the actual web interface.
13:30 dbs As an MFHD serials library, I forget how much "fun" actual serials can be for those performance use cases
13:31 krvmga tsbere: thanks
13:33 Dyrcona Nice thing about running your own is you can have private git repos for those branches that contain passwords. ;)
13:34 krvmga wow
13:34 * krvmga wonders if Dyrcona is pulling his leg.
13:34 tsbere krvmga: Note that private repos don't show up in the web interface at all, and as such can't be looked at there.
13:34 sarabee joined #evergreen
13:34 tsbere But they can be managed by gitolite fairly easily
13:34 krvmga quite interesting
13:34 tsbere The evergreen community has private repos on git.evergreen-ils.org for security fixes, as an added example.
13:35 krvmga this info is all very cool
13:35 tsbere Wouldn't do to reveal a security issue by having the proposed fix sitting there in public, after all ;)
13:37 dbs gogs.io should be even easier to set up, in theory. all depends on what you're looking for
13:43 jeff we've been using a hosted copy of gitlab (managed by ansible, naturally) for internal repos for the past few weeks. so far, so good. i haven't had to unbreak anything.
13:44 jeff ansible playbook, linode account, ssl cert.
13:46 jeff oh hey, gogs. i remember that.
13:47 jeff dbs: you've been playing with go lately, iirc?
13:50 jeff i guess a better question... "so, dbs: go, worth a look for something to play with / learn?"
13:51 berick the mind boggles at the (tapped, I'm sure) potential for Go puns.
13:53 berick yay, SIP is good for something...  https://twitter.com/KCLS/status/664145318829686784
13:53 Dyrcona Well, I stuff my personal, private repos on my FreeBSD box at home with ssh access.
13:53 Dyrcona I put my personal, public stuff on github.
13:54 berick i like bitbucket for private (but not too private) repos
13:55 krvmga one of the skills i want to learn is how to install evergreen (and opensrf) from git
13:55 jeff berick: "sorting trophy"?
13:55 krvmga it seemed to me that being REALLY familiar with repos was a prerequisite
13:55 * tsbere finds it easier to install them from git than from tarballs
13:56 jboyer-isl jeff: RFID sorter races. The sorter needs to know the shelving location to sort the books.
13:56 krvmga krvmga is hoping to have the same experience as tsbere
13:56 berick jeff: our central sorting facility for transiting copies has a yearly competition (that i'm just learning of).
13:56 berick for most items sorted in an hour
13:56 jboyer-isl Or maybe just the branch, I don’t know if you can get shelving locs.
13:57 berick jboyer-isl: transit destination, in this case.
13:57 jeff jboyer-isl: our sorter uses shelving location.
13:57 jeff jboyer-isl: so i can confirm that's one of the values returned. :-)
13:58 jeff rather, the way we use our sorter relies mostly on library branch or shelving location (if at this branch).
13:58 jboyer-isl Excellent. Now I need a reason to keep up with recent RFID tech. (we’re not likely to get it here. :-/ )
13:59 berick we're not even using rfid.  the sorter has a barcode scanner.  low tech, but apparently gets the job done
14:00 berick s/low tech/different tech/
14:02 jboyer-isl berick: That is fast for a barcode scanner. I’ve not heard of that kind of set up. (works, obviously, but I’ve apparently not talked to anyone else doing it.)
14:05 berick you can see the scanner up near the top https://www.youtube.com/watch?v=5Tzl-AhLXFo -- it's like an episode of How It's Made
14:06 berick but more humans than robots
14:06 jeff > Reload Count Values
14:06 jeff Firmware Version:   2.0A         Webware Version: 2.1A
14:06 jeff er, yeah.
14:06 jeff what i was trying to paste:
14:06 jeff > IF YOU CAN ONLY SEE THE DATE, TIME AND IP ADDRESS BELOW,  PLEASE ENABLE JAVA SCRIPTS ON YOUR BROWSER
14:07 berick ENABLE ALL THE SCRIPTS
14:07 jboyer-isl Give me a script please, one script only.
14:08 jeff jboyer-isl++
14:09 jeff i do like the calendar in this interface: NOVEMBER,20015
14:10 kmlussier Stompro / phasefx / mmorgan: Thanks!
14:10 jboyer-isl berick: That is an enormous setup, I don’t envy the person that gets a call when something goes wrong.
14:13 berick yeah, it's a beast
16:04 kmlussier Dyrcona++ #NCIP fixes
16:06 Dyrcona heh..
16:06 Dyrcona NCIP will be the death of me....
16:07 Dyrcona So, latest thing; the lending library sent our borrower the wrong copy, right title, wrong barcode as far as NCIP is concerned.
16:08 mmorgan /me is confused.
16:08 mmorgan Why does NCIP care about the title?
16:09 tsbere mmorgan: I think the issue is more the *barcode*
16:10 tsbere As in "they grabbed a copy off of the shelf, but not the copy they told the system they were sending"
16:11 mmorgan Ok, gotcha.
16:11 Dyrcona Right.
16:12 Dyrcona Can't fix that one with software. Have to tell the library to just use the barcode in the Commonwealth Catalog and not the one on the book.
16:13 mmorgan Seems like that would be rare that they
16:13 mmorgan 'd send a different barcode than the one they enter is comcat.
16:13 jeff in an INN-Reach environment, the DCB client has an "insert barcode" function which is used for some circumstances like that.
16:13 mmorgan Or did they misscan or mistype the barcode?
16:14 jeff It can be used in cases where a lending library ships you an item without actually processing it on their end.
16:14 Dyrcona mmorgan: I don't know, but apparently it is the right title with the wrong barcode.
16:14 jeff Annoyingly, when you use this feature (at least as the borrowing library), NONE of the NCIP messages that would normally be sent are sent. :P
16:14 Dyrcona jeff: We have no provision for that with this vendor.
16:15 jeff So, since there are critical bits of info in those intermediate NCIP messages, Much Breaks.
16:15 Dyrcona Best one could do is cancel the request, place a new one, and ship a different item.
16:15 Dyrcona jeff: I'm not surprised that much breaks.
16:16 Dyrcona It surprises me when much works. :)
16:16 jeff heh
16:17 Dyrcona I am a firm believer in Gilb's laws of unreliability.
16:21 jeff :-)
16:28 kmlussier Bmagic: How does Tuesday, December 15 look for the next Bug Squashing Day? Will you be able to set some Sandboxes up for us?
16:28 kmlussier It's very rare that we have a day when everyone says they can participate, but it happened this time around. http://doodle.com/poll/us66uhf35ym9b766
16:30 jeff kmlussier: if i was interested in contributing some sandboxes, what should i know?
16:30 jeff i'm assuming VMs with concerto and a given branch loaded?
16:30 kmlussier Ooh!
16:30 kmlussier Yes, concerto and a given branch. I try ask people to submit their requests ahead of time to give us time to get ready.
16:31 jeff also, how long do the VMs need to be made available? 24 hours?
16:31 jeff 12 hours? 72 hours? :-)
16:31 kmlussier jeff: We've gotten to the point where we really could use more Sandboxes. I think I used a community demo server last time around.
16:32 kmlussier jeff: For a little more than 24 hours. You'll want to have it in Sandbox mode before you leave the day beforehand, and then you could repurpose it the day after Bug Squashing Day.
16:32 kmlussier jeff: Sometimes, people ask to keep theirs a little longer if they didn't get through testing during the official day. However, the MassLNC Sandboxes can be used after Bug Squashing Day.
16:36 Bmagic kmlussier: sounds good
16:37 kmlussier Bmagic: In that case, Sandbox requests should be submitted by the end of the day Thursday, December 10? That will give you Friday and Monday to prepare. Or do you need it to be a little earlier?
16:38 Bmagic kmlussier: that will be great!
16:38 kmlussier OK, I'll set the date.
16:38 Bmagic and some bugs can share a sandbox
16:39 kmlussier jeff: Let me know if you're interested in loading up some Sandboxes. They would be very helpful. Bug Squashing Day is becoming a popular thing.
16:40 kmlussier jeff: Also, tsbere has some handy scripts for quickly loading up a Sandbox if you're using Ubuntu.
16:40 jwoodard n00b question: What is the "Reserved/Pending" status?
16:41 Bmagic tsbere: re penalties - What is a staff created penalty?
16:41 tsbere Bmagic: Adding via the staff client in the messages tab
16:42 Bmagic tsbere: the issue is a staff member is creating a "note" penalty from the patron screen under "Messages" -> "Apply Standing Penalty"  and I don't see anywhere on the dialog to set the branch or the system
16:43 tsbere Bmagic: That is the kind that you can, system-wide, set a default "depth" of. We set the three defaults to depth 0, then made "System Level" variants for depth 1 (they show up in the dropdown)
16:45 tsbere Bmagic: I believe the ones you would want to set the depth on are STAFF_CHR, SILENT_NOTE, and ALERT_NOTE for the default set
16:45 * Bmagic goes and looks at the interface
16:46 mmorgan tsbere: Bmagic: The depths for those penalties can be viewed and changed in Admin - Local Admin - Standing Penalties, right? Not in the messages area where staff apply them.
16:46 tsbere mmorgan: That sounds correct. Not sure why it ended up in Local Admin instead of Server Admin though...
16:47 Dyrcona One of those things, I guess.
16:49 tsbere Bmagic: Note that changing the default *won't* change any pre-made ones. You have to update the DB to fix them.
16:49 Bmagic ok, so the penalty named "ALERT_NOTE" is the penalty that is applied when a staff member chooses "Alert"  (ID 20 from stock) ?
16:49 tsbere Bmagic: Yes.
16:49 Bmagic Got it. Thanks!
16:49 Bmagic tsbere++
16:50 tsbere Bmagic: SILENT_NOTE being the "Note" and "STAFF_CHR" being the "Block"
16:50 tsbere Bmagic: If you grab the IDs of those three (or whatever ones you do change org_depth to 0 on) you can bulk update the already-made ones with a quick query ;)
16:52 Bmagic org_depth of 0 means it would alert or show for anyone logged in and registered at any workstation, 1 means workstations registered at any of the branches in the system, and 2 would be the specific branch where the alert was created?
16:52 Dyrcona Bmagic: Yes.
16:52 tsbere Bmagic: Generally, yes. In reality it is "create it at the org unit ancestor at that depth" so the system's tree has an obvious effect ;)
16:53 kmlussier ldw: Sorry about that!
16:53 sarabee joined #evergreen
16:54 Bmagic Right on. thanks again!
16:55 ldw kmlussier: not a problem.  Glad to have another participant.
16:55 tsbere Bmagic: Again, feel free to create additional ones for the dropdown in the staff client. We have a version of each with depth set to 1 instead of 0 for "local" variants.
16:56 Bmagic I see that, if it turns out they want more control, I understand how to give it!
17:18 mmorgan left #evergreen
17:21 sbrylander joined #evergreen
18:16 Christineb joined #evergreen
20:48 phillipjphry jeff: Thank you! I'll fire up a debian node with more ram and see how this runthrough goes.
20:56 phillipjphry csharp: I'm using the instructions in the base of the github file, and linked throughout
23:00 phillipjphry http://www.fishyjoes.com/
23:00 phillipjphry got it working, still a few bumps that I need help ironing out

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat