Evergreen ILS Website

IRC log for #evergreen, 2015-05-19

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
02:49 Callender_ joined #evergreen
04:57 pinesol_green Incoming from qatests: Test Success - http://testing.evergreen-ils.org/~live/test.html <http://testing.evergreen-ils.org/~live/test.html>
07:40 sarabee joined #evergreen
07:48 jboyer-isl joined #evergreen
08:00 mrpeters joined #evergreen
08:11 ericar joined #evergreen
08:16 rjackson_isl joined #evergreen
08:21 akilsdonk joined #evergreen
08:22 ericar_ joined #evergreen
08:33 Dyrcona joined #evergreen
08:39 mmorgan joined #evergreen
08:39 graced joined #evergreen
08:51 Shae joined #evergreen
08:52 ericar_ joined #evergreen
09:01 jwoodard joined #evergreen
09:11 Newziky joined #evergreen
09:12 csharp eeevil: I definitely see a difference post-fix for bug 1438136 on my test server (http://next.gapines.org), but when I applied it to PINES production just now, I'm still seeing long-running queries - I'll try and dig up a core query and EXPLAIN ANALYZE it
09:12 pinesol_green Launchpad bug 1438136 in Evergreen "OPAC searching significantly slowed by adding format filters" (affected: 1, heat: 6) [High,New] https://launchpad.net/bugs/1438136
09:14 montgoc1 joined #evergreen
09:17 maryj joined #evergreen
09:18 csharp eeevil: http://pastebin.com/vAD0GMZ9 - whenever you get a free moment ;-)
09:19 csharp it's still underestimating the number of rows to scan for the mrv index
09:20 Newziky left #evergreen
09:21 yboston joined #evergreen
09:30 csharp dbs: if you don't mind, I'm going to work on branch-ifying your (and eeevil's) solution to bug 1206936 to make that pushable into master
09:30 pinesol_green Launchpad bug 1206936 in Evergreen "money.transaction_billing_summary view displays incorrect billing_type and billing_note for the actual last transaction" (affected: 1, heat: 6) [High,Triaged] https://launchpad.net/bugs/1206936
09:37 RoganH joined #evergreen
09:41 jeff @weather ktvc
09:41 pinesol_green jeff: The current temperature in Traverse City, Michigan is 39.2°F (8:53 AM EDT on May 19, 2015). Conditions: Overcast. Humidity: 73%. Dew Point: 32.0°F. Windchill: 32.0°F. Pressure: 30.13 in 1020 hPa (Rising).  Freeze warning in effect from midnight tonight to 8 am EDT Wednesday...
09:41 mmorgan Brrrr...
09:43 csharp @weather 30033
09:43 pinesol_green csharp: The current temperature in Leafmore, Decatur, Georgia is 71.1°F (9:42 AM EDT on May 19, 2015). Conditions: Scattered Clouds. Humidity: -999%. Dew Point: 12.2°F. Pressure: 30.08 in 1018 hPa (Steady).
09:44 csharp argh - now I'm arguing with myself about whether money.transaction_billing_summary and money.transaction_billing_type_summary can have the same view definition
09:45 * csharp doesn't know what each does
09:47 csharp hmm, I don't see either defined in fm_IDL.xml
09:47 csharp obviously they're used somewhere, right?
09:48 pastebot "mmorgan" at 64.57.241.14 pasted "transaction.billing... View definitions" (15 lines) at http://paste.evergreen-ils.org/55
09:50 jeff csharp: they currently differ and you're proposing changing them to be similar/identical?
09:50 rjackson_isl jeff: burned the tops of my feet a bit at Esch Rd Beach on Saturday - it was 75 and mostly calm then
09:52 csharp jeff: right - basically dbs's approach is to use the money.materialized_billable_xact_summary table to populate money.transaction_billing_summary, but if we're going to do that, I think it makes sense to change all of the views that have the flaw in bug 1206936
09:52 pinesol_green Launchpad bug 1206936 in Evergreen "money.transaction_billing_summary view displays incorrect billing_type and billing_note for the actual last transaction" (affected: 1, heat: 6) [High,Triaged] https://launchpad.net/bugs/1206936
09:53 csharp and with that approach, there's not a way to distinguish the two views, except by name
09:53 csharp grepping the code only shows were the views are created in the SQL script, so I'm at a loss to where to look next at where they are actually referred to :-/
09:55 jeff money.transaction_billing_summary may have been unused since 2006, commit ef811b8
09:55 pinesol_green [evergreen|miker] speeding up the open_transaction_summary view - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=ef811b8>
09:57 csharp hmm
09:57 mmorgan money.transaction_billing_summary looks correct in that it's selecting last(billing.billing_type). money.transaction_billing_type_summary does not look correct as it is not selecting the LAST billing type.
09:58 jeff And it's possible that money.transaction_billing_type_summary has never been used since being introduced in 2005, commit 06d7269
09:58 pinesol_green [evergreen|miker] adding "open only" views - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=06d7269>
09:59 csharp mmorgan: see the original description for bug 1206936 for how that's not working as expected
09:59 pinesol_green Launchpad bug 1206936 in Evergreen "money.transaction_billing_summary view displays incorrect billing_type and billing_note for the actual last transaction" (affected: 1, heat: 6) [High,Triaged] https://launchpad.net/bugs/1206936
10:00 csharp wow - ok
10:01 jeff Note my use of "may" and "possible". I'm just going by what I'm seeing in git with basic string searches.
10:01 csharp well, you're confirming my original findings that "ain't nothin' usin' these thangs"
10:01 jeff Also, especially since the views are documented in the schema docs, it's possible that anyone writing SQL reports may have used them.
10:02 csharp yeah
10:03 jeff Not sure if I'd advocate "fix+deprecate, remove" or "remove".
10:03 jeff Since if you were using them they probably were giving data other than that you sought...
10:04 csharp well, the complaint that spawned the bug was that one of our PINES librarians found that the last billing type and last billing note were often wrong when running reports that would otherwise depend on those being right
10:04 jeff Having pulled more chunks of Craftsman out of the repo during my layovers, I'm leaning toward "excise in a precise, documented, but otherwise ruthless fashion"
10:04 jboyer-isl In cases like this I'm a fan of "Don't fix, paste red flags everywhere + fast track deprecation, remove in version now()+1.
10:05 jboyer-isl If it's fixed, people will just keep using it.
10:05 csharp jboyer-isl: yeah, I'm not interested in fixing something that's not used anywhere
10:05 jeff Because is it possible that someone's custom skin was referencing that image that was brought in with Craftsman but not removed when Craftsman was removed? Sure. Is it worth carrying around a whole pile of Craftsman "just in case"? My opinion there is "no."
10:06 csharp this is an example of my not knowing how the front end connects to the back end - I'm now going to have to retrace my steps and figure out how I got here in the first place
10:07 jeff jboyer-isl: since you have "log all queries" for pgbadger purposes, can you grep a month or so worth of production queries to see if those views are referenced? ;-)
10:07 jboyer-isl pgbadger only looks at yesterday's queries, so we only keep a couple weeks worth for situations like this. But I imagine that's enough... ;)
10:08 jeff presumably, as long as we're not missing a generated reference to the view name that only comes into play on the third monday of every month containing a blue moon.
10:09 jboyer-isl Heh, to explain for the logs: pgbadger creates static reports, so it only needs to look at a single day at a time, build that report, and then never look again. It will let you keep as many as you like.
10:09 csharp Classic Open Transaction Summary was the original view with the trouble, according to the ol' helpdesk
10:10 jboyer-isl Ah, "Classic" in the name. Set it aflame, problem solved. ;)
10:11 csharp which pulls its data from money.open_billable_xact_summary
10:12 jeff poetic.
10:12 csharp jboyer-isl++
10:12 jeff stay away from poison ivy by remembering: "Leaves of three? Let it be."
10:13 jeff avoid zombie reporting views by remembering: "Classic in the name? Set it aflame."
10:13 jboyer-isl jeff++
10:13 csharp I like it (or I would if 95% of all PINES reports templates weren't built on such views :-) )
10:14 csharp reports--
10:15 jeff well, right. in this context a "Classic" report is code for "A report that tries to do it the way that PINES was used to doing is so as not to be too new and scary." :-)
10:18 csharp 6b3661b8
10:18 pinesol_green [evergreen|erickson] updated some existing billing views to use the new (faster) money.materialized_billable_xact_summary - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=6b3661b>
10:25 dbwells joined #evergreen
10:33 csharp ok - it's not just the reports issue (which was apparently fixed with the above commit long ago) - when looking at the bills summary for a patron account, the wrong last billing type is showing
10:33 csharp *that's* why dbs got involved, because someone found that there too
10:33 jeff csharp: where are you seeing the wrong billing type?
10:35 csharp for a couple of example patron accounts, in the Bills UI, the summary shows a Last Billing Type when that column is displayed (in one case "Overdue Materials"), but when Full Details is selected, I can see that the *actual* last billing type was "Lost Materials"
10:37 csharp http://irc.evergreen-ils.org/​evergreen/2013-07-31#i_18620 shows a discussion a couple of years ago when the bug was originally filed
10:37 jeff And it isn't a case of "item was marked lost at 14:30 on a day where it had ALSO been billed an overdue fine (in the FUTURE) at 23:59?
10:37 csharp jeff: right - it's not that
10:37 jeff k. always have to check. :-)
10:38 csharp "eeevil csharp: actually, the use of last() there is just to get /any/ billing type, we don't care which ... or, rather, we couldn't care which, inexpensively, when the view was written originally because there were no window functions. there are now, so we can care cheaply"
10:38 csharp but window functions were resource-depleting which is why we're looking at just using the materialized view at this point
10:39 csharp but my plan of attack will only include tables actually in use ;-)
10:39 jeff resource-depleting as in they consumed a bit of ram?
10:46 RoganH_ joined #evergreen
10:50 csharp jeff: well, just bad queries, actually - long-running
10:51 csharp jeff: would you mind helping me locate where "search_money_billable_transaction_summary" is defined? as in "my $mbts = $e->search_money_billable_transaction_summary", line 2269 of Open-ILS/src/perlmods/lib/O​penILS/Application/Actor.pm
10:51 csharp I feel like I'm being led in a circle when grepping (or ack-ing actually) the code for that :-/
10:52 jeff auto-generated methods in CStoreEditor.pm, iirc.
10:52 jeff hold for line number.
10:52 csharp aaahhh
10:53 * berick confirms
10:54 jeff Open-ILS/src/perlmods/lib/Op​enILS/Utils/CStoreEditor.pm in sub "init", near the comment "Load up the methods from the FM classes" around line 917 in master
10:54 csharp jeff++ # excellent
10:59 csharp okay - so that leads me to money.materialized_billable_xact_summary, which is a view of money.billable_xact_summary, which uses the same last() functions that caused the problem in the first place :-/
10:59 csharp so... I'm back to where I started
11:00 jboyer-isl if money.materialized... is a view, issues are piling up rapidly.
11:00 csharp funny, my original thought was "well, looks like this is all sewn up.  I'll just throw it into a branch so we can start the sign-off process"
11:01 csharp jboyer-isl: it's a materialized view
11:01 Dyrcona materialized usually means a table standing in for a view.
11:01 jboyer-isl Also, Re: grepping for money.transaction_billing_summary, no mention in the db logs for the last 12 days.
11:02 jboyer-isl csharp: Oh, but you may be running a version of postgres that has legit materialized views, where mine are still built from triggers. That would make \d+ in psql give us different things.
11:02 csharp jboyer-isl: nah - this is trigger-based - whatever's in core EG
11:03 jboyer-isl Then it should still be a real table, that's what I've got here.
11:03 csharp and this <-- is why that bug has been open for nearly 2 years :-/
11:03 csharp jboyer-isl: yeah - sorry - it's a real table - I should have been more accurate
11:04 csharp I get to a point where I just decide it's too much work for the slender benefit
11:06 jboyer-isl I thought there was some interest in getting core to base the definition of (some of?) our manually materialized views to use the built-in PG stuff at some point, but that may be further down the line.
11:11 csharp I like the way that sounds and I'm up for helping when the time comes
11:14 jeff jboyer-isl: presently in 9.3 and 9.4, postgresql "native" materialized views are only updated on-demand, and in full (not incrementally).
11:16 jboyer-isl jeff: Eeewww... That doesn't sound at all good. All the joys of using a naive garbage collection system, now in database form!
11:17 jeff Also, there's a bug with pg_restore when using materialized views: postgres can attempt to refresh the view (populate it) before indexes are created, which is... slow.
11:23 jboyer-isl This is how the sausage gets made I suppose, new features, new bugs.
11:31 bshum Stompro++ # new docs committer
11:39 jeff Stompro++
11:42 jboyer-isl jeff: you have collectionHQ mail
11:42 jeff jboyer-isl++ thanks!
11:42 jboyer-isl Good luck
11:43 Dyrcona <aol_voice>You've got mail!</aol_voice>
11:44 jboyer-isl Dyrcona understands how I think.
11:54 bmills joined #evergreen
12:01 dkyle1 so ccvm has id 587 but ccaed has no coded_value 587.... seems to be why I can't get a book to properly re-ingest and have a book icon - don't know how it got an ebook icon to begin with
12:02 dkyle1 removing the 587 from ccvm and re-saving the bib did change the icon from ebook to book. anyone know if that will cause other problems?
12:03 dkyle1 and possibly a  bug in the seed data
12:10 * tsbere has little to no clue what dkyle1 is talking about at this point
12:11 * dkyle1 laughs
12:14 tsbere dkyle1: I will say that the icon_format definition for book (which controls when that shows up) says that the item_form can't be one of several values, including those that are required for ebook to show up...
12:15 tsbere dkyle1: From my docs, any of the following item forms will stop the book search filter/icon: a,b,c,f,o,q,r,s (d, for large print, only stops the icon)
12:16 dkyle1 had a book that was displaying with an ebook icon, found that config.coded_value_map row id 587 has no corresponding coded_value entry in config.composite_attr_entry_definition
12:16 tsbere dkyle1: ebooks show up with item_form set to one of o,q,s
12:16 Dyrcona Check the MARC is the usual answer.
12:17 tsbere dkyle1: MVLC's row 587 isn't ebook, it is "item_form=q", which the ebook and book icon_formats check indirectly.
12:17 dkyle1 Dyrcona: indeed, and item_form showed blank in marc editor, but showed as q in metabib.record_attr_flat
12:17 sandbergja joined #evergreen
12:17 tsbere dkyle1: The real solution is to find out where that q is coming from and remove it from the marc.
12:19 dkyle1 tsbere: thanks, yes. I was re-saving the marc but not seeing the expected change in  metabib.record_attr_flat
12:19 dkyle1 until I removed the 587 row from ccvm
12:20 Dyrcona I wouldn't remove the row.
12:21 Dyrcona I think something in the 007 and/or 008 can also affect that, not just the "fixed" fields.
12:23 jihpringle joined #evergreen
12:23 tsbere dkyle1: All that removing the row does is break the item_form indexing.
12:24 tsbere dkyle1: The item_form can come from the 006 or the 008. Where varies by marc type.
12:24 dkyle1 tsbere: I won't leave it out, but for some reason that allowed me to fix that  record
12:25 tsbere dkyle1: Further, the 006 is repeatable and likely isn't represented in the fixed field editor because of that.
12:26 tsbere generally going to be position 23 in the 008 or position 6 in the 006.
12:28 dkyle1 tsbere: so what is the purpose of the config.composite_attr_entry_definition table?
12:29 dkyle1 tsbere++  and I thought I knew a lot about  marc...
12:29 tsbere dkyle1: When a ccvm entry is tied to a "composite" type the definition is in that table
12:29 tsbere composite being "based on other types that would have already been looked up before the composite check"
12:29 tsbere basically
12:30 dkyle1 tsbere: ok, and the definition is where when not tied to a composite type
12:31 tsbere varies. <_<
12:31 kmlussier dkyle1: I don't know if this is helpful at all, but you can see what data is required for the default icons at http://docs.evergreen-ils.org/2.8/_marc_reco​rd_attributes.html#_search_and_icon_formats
12:31 tsbere config.record_attr_definition has a small collection of options
12:32 dkyle1 kmlussier++
12:33 dkyle1 we didn't upgrade for quite a while, so I have a number of new table relationships to fully grok
12:34 kmlussier tsbere++ #For creating that particular piece of documentation.
12:35 tsbere Oh, wow, I forgot I had included "where to find item format by record type" in there.
12:44 Dyrcona Signing out 'cause I'm changing wireless networks in a minute or ten.
12:54 jeff of the two images "progressbar_green.gif" and "progressbar_green-old.gif", both are in use, and the former isn't green anymore. :-)
12:55 jeff Of the 131 image files added by the craftsman skin, 127 are no longer in use and are now slated for removal.
13:06 Dyrcona joined #evergreen
13:08 kmlussier jeff: Not only is it not green, but it's not much of a bar either. :)
13:12 jeff launchpad-- the fastest way to search for something should not be to order by bug id and page back
13:12 maryj joined #evergreen
13:13 tsbere jeff: I usually submit a "new" bug to search. <_<
13:13 jeff yeah. i've done that also.
13:13 jeff of course, it's possible that there isn't a bug when i thought there was.
13:31 berick jeff++ # cleanup
13:38 jeff berick: did you open a bug on removing the legacy self checkout interface? i know there's the branch...
13:38 jeff but can't tell if i can't find the bug because of launchpad or because it does not exist. :-)
13:43 berick jeff: bug 1312297
13:43 pinesol_green Launchpad bug 1312297 in Evergreen "time to remove the old web-based selfcheck interface" (affected: 2, heat: 10) [Wishlist,Confirmed] https://launchpad.net/bugs/1312297
13:43 berick jeff: fyi, i googled 'site:bugs.launchpad.net evergreen selfcheck remove' ;)
13:43 berick only way anything works for me
13:44 * bshum likes the "deprecation" tagged bugs :)
13:45 kmlussier RoganH_++ # http://www.roganhamby.com/evergreen/201​5/5/18/evergreen-conference-2015?utm_so​urce=twitterfeed&amp;utm_medium=twitter
13:48 Dyrcona joined #evergreen
13:58 jboyer-isl csharp: I posted something to the dev and general lists a while ago, is it stuck in a mod queue?
13:58 jboyer-isl I don't know who else has such access to the lists, but I'd welcome help from wherever.
14:12 jeff berick: thanks!
14:41 cmccarthy joined #evergreen
14:54 bshum jboyer-isl: I think csharp is the primary admin for those mailing lists
14:54 * bshum checks the wiki
14:54 bshum http://wiki.evergreen-ils.org/doku.php?id=we​bsite_administration#evergreen_mailing_lists
14:54 bshum Looks like phasefx may have powers too
14:55 bshum jboyer-isl: Maybe they can fix it for you, and make you a moderator too.
14:55 bshum :P
15:02 csharp jboyer-isl: I don't see anything in the moderation queues for either list from you
15:06 cyclops joined #evergreen
15:11 jeff good excuse for me to log in and remember that those lists aren't on the community list host yet. :-)
15:11 bshum At some point we ought to finish the plan and execute moving those over.
15:27 csharp personally, I would just start new lists, "General", "Dev", "Documentation", and the git commit lists on the list.evergreen-ils.org server, have a 30 day "transitional period" where both lists work, then close down the list.georgialibraries.org lists down and move the older archives over
15:28 dbs csharp++
15:28 csharp that way we aren't really trying to migrate anything
15:30 csharp if'n y'all want, we could begin that transition, say June 1 and be fully moved over by July 1 (those dates are weekdays, so no awkwardness around weekends/holidays)
15:31 csharp mailman scripts make mass export/import of current members very easy
15:31 bshum Certainly a good thing.
15:31 bshum My only thought is still back to keeping the name "dev" for that list, and instead using something else more generic.
15:32 bshum Like "technical"
15:32 bshum Just to avoid the whole round yet again about why "dev" is for "developers"
15:32 bshum But anywho.....
15:32 jeff What's the purpose of making the migration take 30 days?
15:47 csharp jeff: it was a nice round number plucked from thin air to account for people who may only occasionally check EG mail?  it's arbitrary, though, so if someone has a better idea, I'm all for it ;-)
15:49 jeff Do you see there needing to be some user action that every subscriber takes during that period of time?
15:49 csharp nah
15:49 csharp we could just subscribe all the current subscribers to each list to the new version
15:50 jeff So it's just a "have two lists that are disconnected from each other for 30 days for maximum confusion" period? ;-)
15:50 jeff infrastructure hack-a-way. my backyard, grill + wifi.
15:50 csharp I just figured people might need time to adjust to the change - I'm open to alternatives
15:52 jboyer-isl I just re-sent it, lets see if it has again run afoul of cosmic rays or something.
15:56 gmcharlt csharp: I personally would prefer advance warning followed by a firm cutoff
15:56 csharp gmcharlt: fine with me
16:00 jboyer-isl Aha, I apparently signed up under my old email (jboyer1@...) and now have to fix that.
16:02 jboyer-isl Oh, Change Globally check box, you are my best friend.
16:07 Dyrcona joined #evergreen
16:08 jboyer-isl I sent that thing 3 times and couldn't be bothered to correct that awful looking run-on sentence. Madness.
16:10 jeff amused at the SSL warnings on jboyer-isl's onedrive.live.com urls
16:11 jeff (SHA-1 cert in the chain and not itself rooted in the trust store)
16:11 jeff you probably won't see if it your browser/os includes the cert in question in its default trust store.
16:12 jboyer-isl I wondered what warnings you might be seeing. Are you on Windows? I don't think I've seen that here.
16:13 jeff standard SHA-1 warning in Chrome, "The site is using outdated security settings that may prevent future versions of Chrome from being able to safely access it."
16:13 jeff red X, no interstitial.
16:15 jeff mostly because my relevant trust store doesn't include "VeriSign Class 3 Public Primary Certification Authority - G5"
16:16 abowling joined #evergreen
16:16 jboyer-isl Oh, those. I see.
16:16 jeff so it has to rely on chaining it to "Class 3 Public Primary Certification Authority", which is in my trust store.
16:16 jeff but that signature algorithm is SHA-1, thus the warning is tripped.
17:07 mmorgan left #evergreen
17:34 abowling left #evergreen
19:57 akilsdonk joined #evergreen
20:16 jihpringle joined #evergreen
21:11 sarabee joined #evergreen
22:32 gsams joined #evergreen
22:34 akilsdonk joined #evergreen

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat