Evergreen ILS Website

IRC log for #evergreen, 2017-08-04

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
02:26 remingtron_ joined #evergreen
04:03 Stompro joined #evergreen
04:05 Stompro joined #evergreen
04:31 book` joined #evergreen
04:31 pinesol_green News from qatests: Test Success <http://testing.evergreen-ils.org/~live>
06:40 rlefaive joined #evergreen
07:13 rjackson_isl joined #evergreen
08:37 mmorgan joined #evergreen
08:45 bos20k joined #evergreen
08:46 remingtron ejk: Did anyone respond to your OpenSRF gateway question from yesterday?
08:48 remingtron ejk: if you have code you can share, I bet someone would take a quick look at it.
08:50 ejk Thanks, not in the office yet but I'll try to find something later this morning. I think the key is figuring out how to format the objects when they are method arguments
08:56 _adb joined #evergreen
09:04 kmlussier joined #evergreen
09:14 yboston joined #evergreen
09:15 Dyrcona joined #evergreen
09:19 Bmagic Am I hearing things or do I remember someone telling me they had a SQL script that really removes rows from biblio.record_entry ?
09:29 Dyrcona joined #evergreen
09:33 bshum Bmagic: Dyrcona wrote one for Bibliomation when I was there, by really remove we mean dropping constraints and getting rid of rows for bibs with nothing interesting attached to them
09:34 bshum So I guess it depends on what you mean by "really remove" too :)
09:34 bshum And for what rows
09:34 Dyrcona What I wrote for Bibliomation only removed bibs with no copies, etc. They had some issues from migration.
09:36 rhamby That's pretty straight forward, just drop the rule protecting deletion.
09:36 rhamby then re-enable it when you're done
09:36 * Dyrcona needs to check the logs.
09:37 rhamby or if you choose to not re-enable it you can live dangerously and see if anything running does deletes instead of updates .... (I don't like living dangrously)
09:37 Dyrcona rhamby: It's not that simple. :)
09:38 Dyrcona There are constraints that prevent deletion, not just the trigger.
09:38 rhamby true, but none are hard to disable
09:38 Dyrcona if you don't also delete everything that references the row, you're going to have serious breakage.
09:39 Dyrcona Well, what happens is, you spend several hours playing whack-a-mole with the various constraints and things that need to be disabled and deleted.
09:39 rhamby sure, but it's not terribly hard, I"ve done it when removing org units and their bibs
09:40 Dyrcona I don't recommend it.
09:40 Dyrcona Storage is cheap. :)
09:41 rhamby if you do have to do it there are steps you'll want to do it in, it's simpler if there aren't volumes or copies but if there are you just have to do some things in particular order
09:42 _bott_ joined #evergreen
09:45 rhamby I thought I remembered it and I do, the Equinox migration repo has org unit removal scripts that contain examples of isolating and removing bibs.  it's for whole org units but could easily be split up and adapted.
09:45 csharp has anyone written a script that parses 'osrf_control --diagnostic' that could be used for nagios-ish monitoring?
09:46 Dyrcona rhamby: I'll check that repository for those scripts later. We might find those handy.
09:47 rhamby Dyrcona: they are, I periodically update them as I do projects.  I just pushed  a few updates yesterday in fact though I admit I mostly update them as I run into things rather than do comprehensive review of possible schema changes.
09:47 Dyrcona I'd like to recommend the migration_tools repo. We use the staged bib loader for our quarterly Backstage updates.
09:47 Dyrcona The above was a general statement for the benefit of the channel. :)
09:50 rhamby :)
09:53 csharp hmm - maybe just 'osrf_control --diagnostic | grep ERR' and alert if it comes back with something?
09:53 Dyrcona csharp: That might work.
10:08 Bmagic Dyrcona rhanby - My reasons is to lighten up the total size for transport. I can't be certain but I have a theory that we could significantly reduce the volume of our cold backups if we weren't tugging a ton of useless data.
10:08 Bmagic It would mean less bandwidth, faster backup times, and just all around easier to work with
10:09 Bmagic The deleted=true represents half of biblio.record_entry
10:11 Dyrcona Bmagic: What it would really mean is less data in metabib.real_full_rec and a smaller metabib_full_rec_index_vector_idx which leads to faster restore times. :)
10:11 Bmagic rhamby: bre rows are connected all over the place, metabib and friends, asset, action, action_trigger, etc.
10:12 rhamby bmagic: yep, but you can delete things in proper order, as I said I've done it and it's even in the scripts
10:13 Bmagic Awesome, where do I go to take a peak? migration_tools ?
10:13 Dyrcona And, I can send you the script I did for bibliomation.
10:14 Bmagic sweet! Thanks guys
10:14 Bmagic rhamby++
10:14 Bmagic Dyrcona++
10:14 Christineb joined #evergreen
10:14 * Bmagic waves at Christineb
10:15 rhamby bmagic: yeah, look at the migration tools repo, it's named something obvioius like 'remove_ou' or something like that, it's a series of scripts i
10:15 Bmagic ty
10:15 rhamby also note that I can not type correctly today /ug
10:16 rhamby if I recall correctly the last time I used them was against 2.11 so there shouldn't be anything major in the schema at that time to roadblock them
10:16 Bmagic even better
10:17 Dyrcona I think it is OK for me to share the script as a gist. It's a few years old so it may miss a few things.
10:18 pinesol_green [evergreen|Bill Erickson] LP#1708510 Postgres 9.6 full text search config - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=aa139de>
10:18 Dyrcona I just did a count and as of 7/31, deleted bibs make up about 1/4 of our bre table.
10:19 Dyrcona I counted on a copy not the actual production database.
10:19 kmlussier Maybe I'm the only one who doesn't know this, but where exactly is the migration tools repo?
10:20 rhamby kmlussier http://git.esilibrary.com/?p=​migration-tools.git;a=summary
10:20 kmlussier rhamby: Thanks!
10:21 mmorgan kmlussier: You weren't the only one who didn't know ;-)
10:21 mmorgan rhamby++
10:25 Dyrcona Bmagic: https://gist.github.com/Dyrcona/​e9b69ac5281502b6755a80447af50504
10:25 Bmagic Dyrcona: TY!
10:30 Bmagic ha! I have 111 rows in bre where id< -1 and deleted='t'
10:30 Dyrcona Bmagic: yeah, I'm not sure how that happens, but I've seen those bib ids < -1 before.
10:31 Bmagic precats?
10:31 Dyrcona At one point, I think I knew what was doing it.
10:31 Dyrcona yeah, maybe precats or something like that.
10:31 berick there's 1 precat bib they all use.  probably not that.
10:31 Bmagic oh yeah, that's -1
10:32 Bmagic starting on -2, the system creates dummy bibs for some reason, which I used to know....
10:34 Bmagic Dyrcona: your select statement identified 859k bibs out of 2 million. That is almost 50% of our bibs! I seriously believe that cutting these out of our database would be a significant improvement.
10:34 Dyrcona Yes, I say that could well be worth it.
10:37 * berick is looking forward to purging a huge pile of action_trigger data
10:37 Dyrcona Instead of using the HAVING clause, I'd probably do that with and <tablename> is null in the where clause today.
10:37 Dyrcona FYI: That script is from March of 2014, so some things have likely changed since then as far as triggers and rules.
10:45 agoben joined #evergreen
10:51 * csharp finds 293,926 bibs with select from Dyrcona's gist
11:20 Dyrcona phasefx++
11:20 Dyrcona phasefx: I answered Eva off list, but I originally thought the message was going to the list.
11:21 Dyrcona Unfortunately, I have not had time to work on Lp 1131238.
11:21 pinesol_green Launchpad bug 1131238 in Evergreen "Move MARC templates into the database" [Wishlist,Triaged] https://launchpad.net/bugs/1131238
11:22 phasefx Dyrcona: thanks man; never enough tuits :)
11:25 Dyrcona Or the tuits are the wrong shape. :)
11:27 StomproJ joined #evergreen
11:31 berick these 4-dimensional hyper-cube tuits are great, but most of the code is inaccessible
11:38 Dyrcona heh.
11:42 Dyrcona "So much time and so little to do. Wait a minute. Strike that. Reverse it. "
11:44 kmlussier joined #evergreen
12:00 khuckins joined #evergreen
12:12 gsams Bmagic: You have more deleted records than I have records total.
12:13 Bmagic gsams: bibs happen
12:14 gsams yeah, we've got about 15% in that boat here
12:30 Dyrcona I haven't actually checked. I'm sure most of our deleted bibs had copies or uris at some time.
12:46 abowling apt-key add -
12:46 abowling ^and it's friday. d'oh!
12:50 Dyrcona Whee!
13:27 yeogai joined #evergreen
13:52 gmcharlt the web staff serials branch now has a pull request; https://bugs.launchpad.net/evergreen/+bug/1708291
13:52 pinesol_green Launchpad bug 1708291 in Evergreen "webstaff: serials module" [Wishlist,New] - Assigned to Galen Charlton (gmc)
13:52 kmlussier gmcharlt++
13:53 csharp created a simple Nagios check for OpenSRF Diagnostics: http://git.evergreen-ils.org/?p=contrib/pin​es.git;a=blob;f=nagios/check_osrf;h=395d107​be79a104e6257efd9d6c45e131d6d4d26;hb=HEAD
13:55 _adb csharp++
13:55 Dyrcona csharp++
14:05 _adb we have a different nagios check in place. i inherited it, i didn't write it. it looks a lot more complicated than yours: https://gist.github.com/anonymous​/5ca7804d3b43aef1c17decbf2448b0c5
14:23 csharp _adb: interesting
14:24 * csharp suspects miker or berick authorship
14:25 * berick takes 2 steps back
14:25 csharp heh
14:26 _adb well, it hasn't caused any problems :-) i have no idea where it came from originally
14:28 _adb https://github.com/evergreen-library-system/OpenSR​F/blob/master/examples/nagios/check_osrf_services looks about right. tsbere++
14:36 csharp tsbere++
14:36 tsbere I remember that. I think we kept having listeners stop listening on us, but to other checks they looked fine.
15:01 khuckins_ joined #evergreen
15:02 sandbergja joined #evergreen
15:14 kmlussier joined #evergreen
15:29 kmlussier joined #evergreen
15:34 StomproJosh joined #evergreen
15:40 khuckins__ joined #evergreen
16:25 gmcharlt The tracking page for next week's feedback fest is ready - https://wiki.evergreen-ils.org/do​ku.php?id=dev:3.0:feedback_fest_2
16:26 Dyrcona gmcharlt++
16:28 abneiman gmcharlt++
16:32 pinesol_green News from qatests: Test Success <http://testing.evergreen-ils.org/~live>
17:04 mmorgan left #evergreen
17:04 gmcharlt https://evergreen-ils.org/evergreen-3-0-deve​lopment-update-13-let-the-fest-begin-again/
19:38 JBoyer_alt joined #evergreen
20:03 remingtron_ joined #evergreen

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat