Evergreen ILS Website

IRC log for #evergreen, 2014-01-22

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
04:14 sseng_ joined #evergreen
04:37 mtate joined #evergreen
06:35 mrpeters joined #evergreen
06:53 csharp @wunder 30345
06:53 pinesol_green csharp: The current temperature in Lakeside, Atlanta, Georgia is 16.9°F (6:45 AM EST on January 22, 2014). Conditions: Clear. Humidity: 50%. Dew Point: 1.4°F. Windchill: 17.6°F. Pressure: 30.13 in 1020 hPa (Rising).
07:05 shadowspar how is the windchill warmer than the actual temperature? =)
07:05 shadowspar @wunder 49783
07:05 pinesol_green shadowspar: The current temperature in Sault Ste Marie, Michigan is -2.2°F (6:55 AM EST on January 22, 2014). Conditions: Light Snow. Humidity: 75%. Dew Point: -9.4°F. Windchill: -18.4°F. Pressure: 30.06 in 1018 hPa (Rising).
07:06 phasefx @wunder 30016
07:06 pinesol_green phasefx: The current temperature in Quail Run - Honey Creek Area, Conyers, Georgia is 19.2°F (7:05 AM EST on January 22, 2014). Conditions: Clear. Humidity: 43%. Dew Point: -0.4°F. Windchill: 19.4°F. Pressure: 30.08 in 1018 hPa (Steady).
07:07 * phasefx would have never known about Sault Sainte Marie were it not for Ticket to Ride :)
07:08 shadowspar heh! yeah, I was surprised to find us on there when I first saw that game =)
07:09 shadowspar iirc back in the 1950s we had passenger rail running from Sault MI west to WI and MN
07:09 timf joined #evergreen
07:09 shadowspar gone now, unfortunately.
07:09 shadowspar & in reality, on the canadian side, the main east-west line completely bypasses us; it goes through Sudbury, 4 hrs east of here
07:10 phasefx they finally removed the train tracks running through my town; it saddened me
08:37 akilsdonk joined #evergreen
08:41 RoganH joined #evergreen
08:55 Dyrcona joined #evergreen
08:55 kmlussier phasefx: Ticket to Ride? The board game?
08:56 Dyrcona The Beatles!
08:57 mmorgan1 joined #evergreen
08:57 mmorgan1 left #evergreen
08:58 rsinger joined #evergreen
08:59 mmorgan joined #evergreen
09:00 Shae joined #evergreen
09:02 * kmlussier has contemplated buying that game many times.
09:02 phasefx kmlussier: I prefer Munchkin, but my extended family likes Ticket to Ride :)
09:03 Dyrcona grr.
09:03 Dyrcona kmlussier: The fine generator is still doubling up on my development vm.
09:04 paxed muchkin. eww. it's one of those games you really start hating.
09:04 paxed +n
09:04 paxed :P
09:04 kmlussier Dyrcona: Bah! Sorry to hear it. :(
09:05 Dyrcona I'll have to figure out what is really going on, instead of relying on hunches. :p
09:05 * csharp thought of the Beatles song too
09:06 * Dyrcona started playing .oggs of Abby Road.
09:06 csharp Dyrcona++
09:06 paxed on boardgamegeek, munchkin is rated 1925th...
09:06 Dyrcona sp... Abbey
09:08 Dyrcona kmlussier: The fine generator ran last night, and I just ran it manually for xact 16718572. It has two entries for each date. :(
09:08 phasefx munchkin is as close as I can get to having my wife play D&D, so it wins for that alone
09:10 jcamins Ooh, Ticket to Ride is intended for 2-5 players.
09:10 jcamins Playing Munchkin with only 2 players is not always very successful.
09:10 jcamins The games are either really really fast or really really slow.
09:10 paxed (really, check boardgamegeek - it's the best place for boardgames/cardgames/etc info)
09:11 phasefx need some wheel'n and deal'n with Munchkin, and the end-game is the best part
09:11 phasefx my least favorite board game: Monopoly
09:11 paxed for a game that's supposedly fast, it tends to drag on and on... or perhaps it's just the group i play with.
09:13 kmlussier Fast? I've never thought of Monopoly as a fast game.
09:13 phasefx want fast, there's slap jack :)
09:13 jcamins I've always thought of it as an incredibly slow game.
09:13 paxed (munchkin, not monopoly. i won't touch monopoly with a 10 ft stick...)
09:26 Dyrcona paxed: Cashflow is better than Monopoly by a long shot. Actually teaches money managment skills, instead of just how to be a dick.
09:27 collum joined #evergreen
09:27 * Dyrcona is now doing a parallel ingest because he has no idea if the one he scheduled last night actually did anything.
09:29 * Dyrcona believes the world is currently run by the kids who used to cheat at Monopoly.
09:30 jcamins Gosh, there are so many fun-sounding games.
09:30 Dyrcona jcamins: The most fun game is WTF! I broke the Fine Generator!
09:30 Dyrcona ;)
09:31 jcamins Dyrcona: "The Do-it-yourself library management game for the entire family"?
09:31 Dyrcona :)
09:31 Dyrcona jcamins++
09:31 graced My new favorite fast game is We Didn't Play Test This At All.
09:32 jcamins graced: lol
09:32 graced I think one of our rounds was over in 10 seconds.
09:32 dbs Monopoly: that's when you check out all of the latest Stephen King / George R.R. Martin copies to yourself
09:32 jcamins graced: This looks hysterical.
09:32 graced it really is
09:33 graced I highly recommend it
09:33 RoganH graced: Cards Against Humanity?
09:33 csharp heh "Integrates with:
09:33 csharp We Didn't Playtest This Either
09:33 csharp "
09:34 graced RoganH: I do love Cards Against Humanity
09:34 phasefx Cards Against Humanity is probably my favorite party game :D
09:34 graced And the phone app  Cards Against Manatees
09:34 paxed assuming "party game" means "needs to be played while intoxicated", then yes.
09:35 phasefx We Didn't Play Test This, would be my 2nd.  "Ah.. Zombies"
09:35 graced Ah Zombies gets me every time for some reason
09:36 phasefx graced++  I had forgotten the name of that game; I'm glad you mentioned it
09:36 graced phasefx: we should play it at lunch on this Friday
09:36 kmlussier graced: Oh, that's a real game! I thought you were making a commentary on untested code. :D
09:36 graced Ha!
09:36 graced No, it's a realy game.
09:36 phasefx graced: the not pointing rule gets me :) especially when I point out someone else's rule violation :D
09:37 * kmlussier now looks to see if there is really a broken fine generator game.
09:37 jcamins kmlussier: yeah, I thought that too.
09:37 jcamins But I looked it up anyway because I'm pretty gullible, and discovered that in this case I should've been more gullible.
09:38 graced jcamins: it is a really unbelievable name for a game
09:44 afterl joined #evergreen
09:47 mrpeters i was invited to play Cards Against Humanity on friday....am I in for some fun?
09:50 eeevil mrpeters: yes. put on your laughin' pants, man
09:51 mrpeters awesome, my old college roommate and his wife were hyping it up big time
09:51 mrpeters and my wife is always begging me to play bored (sic) games hahaha
09:52 jcamins mrpeters: but leave your politically correct hat at home.
09:53 mrpeters lol, i dont have one of those
09:53 jcamins In that case you'll be fine.
09:53 mrpeters ;)
09:53 mrpeters look forward to it now
09:55 jeff grabbing 0854
09:58 rsinger joined #evergreen
09:58 mtiernan joined #evergreen
09:58 mtiernan left #evergreen
10:02 jwoodard joined #evergreen
10:16 dbs @wunder sudbury, ontario
10:16 pinesol_green dbs: The current temperature in Sudbury, Ontario is -22.0°F (10:00 AM EST on January 22, 2014). Conditions: Mostly Cloudy. Humidity: 69%. Dew Point: -29.2°F. Windchill: -40.0°F. Pressure: 30.17 in 1022 hPa (Falling).
10:16 ericar joined #evergreen
10:17 misilot joined #evergreen
10:18 misilot left #evergreen
10:19 kbeswick joined #evergreen
10:21 dbs kbeswick: hey, are you by any chance developing your "find similar" stuff against Evergreen?
10:24 kbeswick dbs: nope, but it could probably be easily integrated if anyone was interested. i am using Solr & python/flask for the backend.
10:25 jeff Dyrcona++
10:25 Dyrcona jeff++
10:25 kbeswick its built to work with our catalog specifically right now
10:25 kbeswick but easily generalized
10:26 dbs kbeswick: cool. would be an interesting proof of concept at least :)
10:26 pinesol_green [evergreen|Dan Wells] Add new option for max/min lost item pricing - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=e42e9b0>
10:26 pinesol_green [evergreen|Remington Steed] Release notes for lp1207903: new lost billing min/max setting - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=e9c5319>
10:26 pinesol_green [evergreen|Jeff Godin] Stamping 0854: data.min_max_prices - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=9f00f09>
10:32 * dbs grumbles again about PostgreSQL XSLT changes
10:34 * dbs also grumbles about out-of-the-box OpenSRF/Evergreen invoking OOM killer on a headless system with 2GB of RAM
10:34 jeff something other than the removal of support for includes (which I was using but probably shouldn't have been)?
10:34 * jeff looks
10:37 dbs https://bugs.launchpad.net/evergreen/+bug/1243023
10:37 pinesol_green Launchpad bug 1243023 in Evergreen "Browse catalogue titles are doubly escaped?" (affected: 1, heat: 6) [Undecided,New]
10:48 kmlussier-at-lib joined #evergreen
10:52 rfrasur joined #evergreen
10:54 jtaylorats joined #evergreen
11:14 bshum Drat
11:14 bshum The selfcheck interface doesn't seem to know how to do password lookups via LDAP instead of the regular passwords in Evergreen
11:14 bshum If using the require password for selfcheck library setting
11:29 RoganH joined #evergreen
11:35 jtaylorats Anyone up for a question about migrating some records...I'm using these instructions....http://en.flossmanuals.net/evergreen-in-ac​tion/sending-gentle-reminders-to-your-user​s-setting-up-notifications-and-triggers/
11:35 jtaylorats ...but getting an error about "function oils_xpath"
11:35 jtaylorats say "No function matches the given name and argument types. You might need to add explicit type casts."
11:36 jtaylorats when I try to run the staging_importer function that it creates.
11:37 tsbere That.....is one of the most misleading urls I have seen.
11:37 jtaylorats I agree.
11:37 jeff huh. the url -- yeah.
11:38 jtaylorats I've searched for oils_xpath and get lots of hits but so far none that related to importing bib records.
11:38 jtaylorats ...at least as far as I can tell.
11:38 tsbere jtaylorats: Perhaps your database didn't set up properly?
11:39 jtaylorats Anything is possible.   I'm having them do the refresh command to reinitialize everything
11:39 jtaylorats ...or whatever it is.
11:40 jtaylorats basically rebuild the schema.
11:40 rfrasur eeevil: nice article
11:40 jtaylorats Was hoping someone had seen this and could offer some insight.
11:40 jtaylorats Yes.  Is a nice article, if I can make it work :-)
11:41 tsbere I have only seen that kind of error when a db load did not work properly, either fresh load or backup. Usually due to missing contrib stuff or issues on loading over an existing DB that didn't drop properly due to an existing connection to it.
11:42 jtaylorats Okay.   Maybe I'll hold off until the rebuild the schema and all the data is gone.
11:42 jtaylorats Maybe there is some type of conflict???
11:44 dbs jtaylorats: umm, what version of postgresql
11:44 * dbs fears postgresql and xpath these days
11:45 jtaylorats "PostgreSQL 9.1.9 on x86_64-unknown-linux-gnu, compiled by gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3, 64-bit"
11:46 dbs jtaylorats: well, that at least should be fine. so I'll +1 tsbere's comment :)
11:46 eeevil rfrasur: thanks :) ... there's a nice restatement of it, from a former coworker, at http://benostrowsky.wordpress.com/2014/0​1/22/of-things-to-come-the-tldr-version/
11:47 tsbere jtaylorats: I would keep the output of the DB load. Redirect it to a file or pipe it into tee if you want to watch it while it goes to the file. Then check to see if any errors happened.
11:47 csharp jtaylorats: what is the output of SHOW search_path ;?
11:48 jtaylorats ""$user",public"
11:48 rfrasur Ben's cool.  Reading his "what I think he meant was" version.
11:50 rfrasur Hah!  Love the last line "Long-range plans are a big deal for Open Source..."
11:50 jtaylorats I've done a direct SQL insert as a test before I use the Copy option.
11:51 jtaylorats Insert worked fine.   It is when it goes to move it to the biblio.record_entry table that it triggers the failing functions.
11:52 dbs huh, the actual notifications doc url is at http://en.flossmanuals.net/evergreen-in-act​ion/sending-gentle-reminders-to-your-users/
11:53 jtaylorats Don't fix it...I may never find it again ;-)
11:53 dbs jtaylorats: "\df oils_xpath" doesn't show anything?
11:53 * dbs realizes he wrote the "Migrating" chapter that jtaylorats is reading :)
11:54 jtaylorats So YOU are the "expert", eh?
11:54 rfrasur dbs++
11:55 jtaylorats I assume that needs to be run from the command line?
11:57 dbs jtaylorats: from psql prompt
11:58 jtaylorats Hmmmmmmmmm...haven't used that yet....I assume I have access to it somewhere here....one moment.
11:58 jtaylorats Been working with the data and not figuring out the particulars of getting it where it needs to go.
11:58 jtaylorats and now, that is.
11:59 dbs jtaylorats: psql -U evergreen -h localhost -d evergreen # adjust according to your hostname / username / databasename
12:02 jtaylorats says (0 rows)
12:02 jtaylorats So does that mean someone didn't install something that is needed?
12:03 jtaylorats I'm late to this project and still figuring out whats what.
12:03 jeff do you have any results for the following? \df evergreen.oils_xpath
12:04 jeff (in a normal system you would not, iirc)
12:05 jtaylorats Interesting...I have 2 rows for that command in "evergreen" but they created a copy named "evergreen-dev" and there are zero rows there.
12:06 jeff when you say "evergreen" and "evergreen-dev", you're talking about two different database names?
12:06 jtaylorats yes.
12:06 jtaylorats I asked them to make a copy so I didn't corrupt the production version.
12:07 finnx joined #evergreen
12:07 jboyer-isl I've noticed errors when doing a pg_dump/restore related to oils_xpath. I wonder if that would explain it?
12:07 jboyer-isl It's not that reassuring for a full dump/restore to complain about things. :/
12:08 jtaylorats Is there something I need to install on the copy database so these functions exist?
12:09 jtaylorats I'm with jboyer...odd a copy wouldn't duplicate everything.
12:09 jboyer-isl I'm not the one to ask, unfortunately, but I'm certainly interested in the resolution to your problem.
12:09 csharp jtaylorats: I think you just need to set the search path so it includes the "evergreen" schema
12:10 csharp jtaylorats: SET search_path TO evergreen, public;
12:10 jeff jboyer-isl: there are limitations with search paths in postgresql that result in at least two stock indexes failing at pg_restore time under common circumstances, due to the way that pg_restore sets search paths, and how we make use of them.
12:10 smyers_ joined #evergreen
12:11 jtaylorats Set that from the evergreen-dev schema?
12:11 jeff jboyer-isl: but that said, if the errors you've seen are those indexes failing to build (which can then be built after the pg_restore completes), it's likely unrelated to the issue jtaylorats has (though jtaylorats' issue might also be search path related :-)
12:11 jboyer-isl jeff: That would explain why our upgrade testing db is not performing as fast as expected.
12:11 csharp jtaylorats: (scrolling up) so "evergreen-dev" is a copy of your production database?
12:12 jtaylorats Yes.
12:12 csharp jtaylorats: then yes, try that
12:13 jtaylorats ********** Error **********
12:13 jtaylorats ERROR: function oils_xpath(text, text, text[]) does not exist
12:14 jtaylorats SQL state: 42883
12:14 jtaylorats Hint: No function matches the given name and argument types. You might need to add explicit type casts.
12:14 jtaylorats Context: PL/pgSQL function "extract_fingerprint" line 34 at assignment
12:14 jtaylorats PL/pgSQL function "fingerprint_trigger" line 10 at assignment
12:14 jtaylorats SQL statement "INSERT INTO biblio.record_entry (marc, last_xact_id) VALUES (stage.marc, 'IMPORT')"
12:14 jtaylorats PL/pgSQL function "staging_importer" line 5 at SQL statement
12:14 jeff jboyer-isl: does your test db have an authority.by_heading index?
12:14 jtaylorats The error has changed a bit.
12:14 jtaylorats It showed more detail before.
12:15 jeff jboyer-isl: the indexes authority.by_heading and authority.unique_by_heading_and_thesaurus are the common ones to fail at pg_restore time due to search_path issues.
12:16 jeff jboyer-isl: i make no claim that those are missing, or if they are that their absence is causing your observed slow performance. :-)
12:17 jboyer-isl jeff: a simple \di authority.by_heading says no. I'll see about re-building them and see what that accomplishes.
12:17 jboyer-isl thanks
12:20 jtaylorats For full disclosure...I changed the RETURNS NULL AS to RETURNS INTEGER AS in the create functions statement.
12:20 jtaylorats Couldn't ever get it to create the function with NULL.
12:21 jtaylorats This line from the document....CREATE OR REPLACE FUNCTION staging_importer() RETURNS INTEGER AS $$
12:21 jtaylorats Just in case that would make a difference.
12:21 ericar joined #evergreen
12:25 jtaylorats Also, in case it was a good thing to try....after setting the search path I did the \df again on the copy and it still says zero rows.
12:27 dconnor_ Does anyone know how to alter the header of an opensrf call?  I'm trying to add 'Accept-Encoding: gzip, deflate' to an opensrf header in copy_browser.js.
12:27 dbs RETURNS INTEGER AS should be fine
12:28 jtaylorats Thought so but didn't want anyone assuming one thing over another.
12:31 dbs That error makes it sound like you're inserting an array of MARC TEXT values rather than a single MARC TEXT value into a given biblio.record_entry row
12:33 jtaylorats the df command shows 2 rows and one of them has an array as the 3rd parameter for that function.
12:33 jtaylorats for evergreen....not the one I'm using.
12:34 jtaylorats I guess I need to try importing into the production database and see what happens.
12:34 jtaylorats If it works there and not in a copy, what would that tell us?
12:35 dbs jtaylorats: that, as jeff and jboyer-isl mentioned, the PostgreSQL dump / restore results in some problems (like this)
12:36 jtaylorats Hmmmmmm...so how does one have any assurance that if their database failed they could recover?
12:37 csharp jtaylorats: other than this? http://www.postgresql.org/d​ocs/9.1/static/backup.html not much ;-)
12:37 dbs Use pg_basebackup instead of pg_dump
12:37 jtaylorats or is this a separate issue?
12:37 jtaylorats Okay.
12:37 dbs http://en.flossmanuals.net/evergreen-in-ac​tion/sending-gentle-reminders-to-your-user​s-setting-up-notifications-and-triggers/ has a lovely cialis spam comment
12:37 jtaylorats I'll pass that on.
12:38 jtaylorats So cialis might be the problem?   ;-)
12:39 jtaylorats Well, sounds like I need to have them create a copy using a different method, assuming they did a dump/restore?
12:39 jtaylorats That sound right?
12:40 Dyrcona Oh. I was gonna say that we run a little script after pg_restore that fixes your problem.
12:40 jtaylorats then check to see if that function exists in the test database?
12:40 jtaylorats Works for me.
12:40 jtaylorats What script?
12:41 jtaylorats Will it insure that all the functions exist?   I assume this problem could exist on many fronts???
12:41 dbs Dyrcona: we could add that script to http://en.flossmanuals.net/evergreen-in​-action/care-and-feeding-of-evergreen/ for the logical backup section!
12:42 pastebot "Dyrcona" at 64.57.241.14 pasted "fix_database.sql" (6 lines) at http://paste.evergreen-ils.org/56
12:43 jtaylorats Is this related to my problem or jeff's?
12:44 jtaylorats I seem to have a missing function.
12:44 tsbere That is kind of a "we don't use the user 'evergreen' as an actual DB user name, so our restore fails to create some indexes, and we need to set the search path anyway...." solution
12:47 jtaylorats So does it related to my problem?
12:47 tsbere I dunno, I wrote the script in question. I kindof lost track of where you are in looking at your problem.
12:47 tsbere <_<
12:47 jtaylorats Me too :-)
12:48 tsbere The search path part may help, though
12:48 jtaylorats My last quest was whether making a copy in a different fashion would solve the problem of missing functions?
12:48 tsbere They may not be missing, just not where the system is looking for them with the incorrect search path
12:49 jtaylorats I set a search path but it still shows zero rows.
12:50 jtaylorats Not sure how postgresql does it but seems odd that if you make a copy of something that it acts different than the original.
12:50 eeevil jtaylorats: you set it in one session, unless you used "ALTER DATABASE" and then restarted services
12:50 jtaylorats Understood...but should I be able to see the function in that session?
12:51 eeevil if it was properly restored, yes
12:52 eeevil if you don't see it, it's time to start over with the restore, having set the search path /before/ restoring
12:52 jtaylorats ...and that goes back to my question...do I need to make a copy in a different way so that it is actually a copy and not part of a copy?
12:52 jtaylorats So use pg_backup rather than dump/restore?
12:53 eeevil to bring all the data into IRC, you used pgdump, correct?
12:53 tsbere We use dump/restore, across servers even, but run that fixer afterwards (and some other local things, depending on what the copy is for...)
12:53 jtaylorats I didn't do it...I'm trying to figure out what I need to tell the people who handle the server.
12:54 jtaylorats Not sure what they did.
12:54 jtaylorats ...for the current copy.
12:54 csharp jtaylorats: you might use the program 'script' to record screen output when pg_dump-ing/pg_restore-ing - you can also add a '-v' parameter to both to see all messages about what's happening
12:54 eeevil then I suggest you start with a fresh dump. pgdump is fine, just make sure that, after you crate the database into which you will restore, that you set the appropriate default search path
12:55 csharp but errors should be printed on the screen as they happen
12:55 jtaylorats okay....would that be....ALTER DATABASE :db_name SET search_path=evergreen, public, pg_catalog;
12:56 tsbere With the proper :db_name or -vdb_name=realname command line parameter to psql
12:57 eeevil background: pgdump dumps data from within a database, so does not capture database creation and setup commands. pgrestore (or just using the SQL script, if that style of dump was used) assumes an existing and properly configured db into which it can restore.  it's a scoping misunderstanding (mismatch between assumption and reality of the way the tools work) that is at fault here. ISTM that documentation is the right fix, if it doesn't already exis
12:58 jtaylorats ISTM?
13:00 dbwells It Seems To Me
13:00 eeevil it seems to me
13:00 jtaylorats okay...knew it was something but my brain is distracted by my stomach and realized I just missed lunch.
13:02 jtaylorats Well, guess I'll stop bothering everyone even though I'm still pretty much clueless about a solution.   Sounds like the restore didn't restore everything but maybe because the setup of the new database wasn't correct, even though I would think database related functions would be in the database.
13:02 jtaylorats So if they create a new database and set a search path and then do the restore it might fix the problem about missing functions?
13:03 tsbere jtaylorats: We run the create database script outright to make the database (and load extensions) then restore into that instead of doing the restore into a non-existant database
13:04 tsbere (the "existing and properly configured db" part being handled by the create database script)
13:04 jtaylorats okay.
13:04 Dyrcona we still have to fix the search path after, though.
13:04 tsbere With the "set the search path and fix those two indexes that never restore for us" step happening after the restore
13:04 jtaylorats That gives me something a bit more concrete to pass on to the server guys.
13:04 jtaylorats Thanks all...sorry for being so dense and ignorant.
13:05 jtaylorats The ignorant I might be able to fix but not sure about the dense part.
13:07 dconnor_ Can you change the header in an opensrf call (I'm in copy_browser.js)?
13:07 jeff dconnor_: which header are you hoping to change?
13:08 dconnor_ jeff: The opensrf call is a new one, but the goal is to pass a gzipped object to make things move quicker (and so I can pass larger datasets)
13:09 dconnor_ jeff: So I need to add  'Accept-Encoding: gzip, deflate' to the header
13:11 dconnor_ I've got the perl doing what I want, but when I call it from the javascript it fails because it's not excpecting a zipped format
13:12 kbeswick joined #evergreen
13:12 jeff dconnor_: it shouldn't fail, though i can see where it might fail to be compressed.
13:13 jeff dconnor_: part of the problem sounds like you're forcing a response to be compressed even when the client hasn't indicated a preference / ability to accept compression.
13:14 jeff dconnor_: i haven't put a lot of thought into this yet, but it seems to me that leaving compression up to the web server or frontend and giving the opensrf javascript request/network bits a way to set an appropriate Accept-Encoding header would be the path forward.
13:17 tsbere dconnor_ / jeff: Perhaps the issue is actually at the apache level. Do we ever add DEFALTE to the INPUT filters, compared to the output ones?
13:18 Dyrcona IIRC, didn't a patch once explicitly remove DEFLATE?
13:20 bshum 9222fb95a4e33eb8e634cf240c6db59e3c1e7349
13:20 pinesol_green [evergreen|Thomas Berezansky] Disable mod_deflate on XMLENT filtered files - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=9222fb9>
13:22 Dyrcona bshum++
13:22 Dyrcona That is what I was thinking of.
13:23 dconnor_ Ok, so is there a way to enable it for a single call?  Or should I just go forward with trying to manually zip and unzip?
13:24 tsbere You could try adding it to the gateway/translator url. It shouldn't have any effect on non-compressed requests.
13:26 dconnor_ tsbere: how would I go about adding that?  From my perspective network.js is a little bit magical and I'm not sure how to change the URL.
13:27 tsbere dconnor_: This would be apache config stuff
13:28 tsbere SetInputFilter DEFLATE I think, if we don't have any others on there in the config
13:28 dconnor_ ok, I gotchya
13:28 dconnor_ I'll poke around and see what I can work out
13:34 jeff My recommendation is to work within the magic of network.js and/or other code to give the option of compression to all opensrf requests, not just having a single opensrf call that always responds with compressed data and needs to be handled specially on the client side.
13:35 jeff It seems more useful in the long term, but does involve potentially more up-front work.
13:39 tsbere jeff: That is part of why I recommended looking into the input filter option/issue, let Apache handle the compression before handing it over to opensrf at all
13:40 jeff tsbere: yes, which would then just require client-side support (which might be as simple as sending the correct request header)
13:40 jeff but of course there's potential for Other Issues :-)
13:51 bshum csharp: I think your new bug is a duplicate of https://bugs.launchpad.net/evergreen/+bug/1259231
13:51 pinesol_green Launchpad bug 1259231 in Evergreen "No email address message stuck in staff client" (affected: 1, heat: 6) [Medium,Fix committed]
13:52 bshum Which is fixed for 2.5.2
13:55 csharp bshum: excellent!
13:57 tsbere That was a simple fix
13:58 csharp tsbere++
14:01 smyers__ joined #evergreen
14:09 bshum Interesting...
14:09 bshum ehardy's message to the list about not seeing the copy location filtering got me pondering that.
14:10 bshum And I think it boils down to disabled JS bits.
14:10 ericar joined #evergreen
14:10 bshum The want_dojo variable stays set to 0 unless autosuggest/googlebookspreview/novelist are enabled.
14:10 bshum Just strange configuration interdependency issues.
14:12 * bshum goes to actually eat his lunch...
14:12 tsbere bshum: could also be "their config.tt2 doesn't say it should be there at all"
14:14 bshum tsbere: True faulty templates do muck things up too :D
14:15 bshum But my casual inspection of the page source code made me more suspicious about the lack of some JS file includes.
14:16 sseng joined #evergreen
14:18 snowball joined #evergreen
14:30 sseng joined #evergreen
14:34 smyers joined #evergreen
14:44 phasefx jeff: are you still doing anything with bounced emails?
14:44 jeff i've been invalidating them. sometimes by hand, sometimes by piping emails to a script.
14:45 jeff for a majority of bounces, i can detect and auto-invalidate
14:45 jeff others should be examined by a human
14:46 jeff i'd like to make that human examination part of a larger crm-like interface
14:46 * tsbere has noticed some vacation messages that look like bounce messages to his auto-filters lately :(
14:47 jeff mailman includes some python for pattern matching bounce messages from common sources. for the most part, i let exim logs drive that.
14:47 tsbere This is basic filtering into folders for my inbox. Said filtering doesn't look at entire messages, just a few headers.
14:47 jeff but a python or perl library for "common bounce messages" would be useful (and might exist, though might suffer from stale updates)
14:48 jeff i'm somewhat disappointed that proper DSN support never took off outside of sendmail.
14:48 tsbere The fact that I am getting "vacation" messages from mailer daemons is an entirely different issue
14:48 jeff phasefx: thinking more on this subject lately?
14:49 phasefx jeff: a library of ours asked if it was possible.  Was curious if you had made it easy :)
14:51 eeevil jeff: obviously the solution is to use sendmail
14:51 * eeevil would if it wouldn't cause his ops folks to put out a contract on him
14:52 * jeff grins
14:52 jeff if the final MTA in the chain is exim, this gets you most of the way there:
14:52 jeff grep -F '**' /var/log/exim4/mainlog | awk '{print $5}' | sed 's/:$//' | sort | uniq
14:52 eeevil M4-EVER!
14:52 jeff (obviously you'll need to adjust that if you're piping syslog elsewhere, etc)
14:53 phasefx jeff: thanks man
14:54 tsbere I'd be more likely to have fun with return paths. Perhaps with "bounces+patronid@yourdomain" and pipe that into a command to say "this patron just bounced" type deal.
14:55 tsbere Perhaps going further with some way to say "email" vs "sms" bounces, but meh. Haven't given it enough thought.
14:55 jeff going the VERP route expands the net of what is caught, but also catches lots of things that aren't "bounces", per se.
14:56 tsbere jeff: what else do you see going down the return path normally?
14:57 jeff larger tangent: I'd like evergreen or something using evergreen to support inbound email. Staff peruse a queue of messages, and for those where a patron has replied to a courtesy email with an "can I renew these?", one or two clicks and an optional personal message later, and poof -- they're renewed, the relevant details are included in an email response to the patron, and you're on to the next message.
14:58 jeff "Please DO reply to this email, it IS sent from a mailbox that is well-attended, and we're happy to help!" :-)
14:58 phasefx jeff++
14:59 jeff tsbere: tempfails, auto-responders, challenge-response (these I'm tempted to classify as bounce / invalid email, etc)
15:00 * tsbere has never seen an auto-responder use the return path, though he can see tempfails as a concern
15:00 Dyrcona jeff: You've invoked Zawinski's law. :)
15:02 tsbere jeff: Regardless, my concern is more along the lines of ensuring we know who bounced. We may still want to check the message itself. We have previously had "this address keeps bouncing but there is no patron with it!" reported to me and I eventually determined the patron's listed address was forwarded. We only saw the bounce from the final destination...
15:02 * dbs has pondered adding JSON-LD supporting a renew action into the outbound pre-overdue emails. but that would require sending HTML mail (yuck) and most MUAs wouldn't do anything with it
15:03 jeff tsbere: in our case, and this came up at least once during the contact invalidation development, if the address bounces, we want to invalidate that email address on all patrons who had the email address listed.
15:04 jeff speaking of email, i had staff thank me for tightening up the email validation regex, both because the data is cleaner now, and because it helps them catch their own typos.
15:04 tsbere jeff: So compare addresses, pull the address out of the DB and search for others with it, etc.
15:05 jeff dbs: gmail would do something with it, though -- and lots of our patrons use gmail. :-)
15:05 tsbere invalidate the one that bounced *and* the one on the patron record, even
15:05 jeff tsbere: but if you start with the email address, the API accepts a "invalidate this email address on any patron that has this email address set"
15:06 tsbere jeff: Oh, and for fun, I have had messages bounce and the only thing I was told was "this message failed" - I had to go to my own sent to find out the address I sent it to. :(
15:06 jeff tsbere: which is why i'm using the exim logs -- they contain the email that we used. :-)
15:07 jeff tsbere: but if we went with VERP and handled delayed bounces, etc, I'd probably encode the email address we have in the envelope.
15:07 tsbere jeff: And how hard is it to say "fetch email from this patron by id, then pass it into said API, perhaps with the inclusion of the final bounce" ;) (also, the bounces I am concerned about are also the backscatter potential ones that are delevered from elsewhere)
15:08 dbs jeff: yes, ours too (now that our uni is cutting over to google apps). went so far as to send myself a test message to play with it. But OTOH I'm still morally opposed to HTML email :)
15:10 * tsbere also hates services that will strip anything that looks like one of their emails from the message, removes most of the headers from the message, and then wonder why you can't do anything to stop them from showing up again later, especially when multiple people got it and only one wants it to stop
15:12 jeff yup. email.
15:13 jeff dbs: and, contrats on groupwise -> google apps :-)
15:13 jeff dbs: or do just the students get google apps?
15:13 dbs Just the students; IT was afraid of kneejerk reactions from faculty
15:17 rfrasur (oy...the disconnect between student and faculty technology always helps the two groups interact better.  Right?)
15:17 jeff rfrasur++
15:17 * rfrasur backs away from the "technology in education" discussion before it gets bloody.
15:18 jeff local school district has students in google apps, staff in exchange. after some unknown amount of time they started giving staff accounts in the "student" domain on google apps because there was demand for use in sharing google docs, etc.
15:18 rfrasur shocking
15:18 dbs rfrasur++ # yep, sure wouldn't want to collaborate or share presentations or anything.
15:19 jeff it's easy to say "that's silly", but i do not envy their inertia and footprint. lots of users.
15:19 rfrasur I just recently filled out a school improvement survey for my kids' high school and am desperate for someone in administration to contact me.
15:19 stevenyvr joined #evergreen
15:20 rfrasur piecemeal technology policies/plans that don't actually address the WHY of using technology and just think you can throw a bunch of software and hardware at something and it'll magically all work.
15:21 rfrasur dbs: do you have to deal with faculty that want students to use technology that they themselves don't use with any level of competency?
15:21 dbs rfrasur: nothing springs to mind, thankfully
15:22 * dbs doesn't want to rip off that scab
15:22 rfrasur That's a good thing.
15:22 rfrasur the "not springing to mind" part...that's the good thing.
15:24 * rfrasur goes back to looking for cheap toddler sing-along cds.
15:28 mrpeters quickie git question -- git diff --stat tag1 tag2 ---- the number after the |  does that indicate the line number at which the insertions/deletions start?
15:29 senator mrpeters: no, the number of lines changes
15:29 senator *changed
15:30 senator as in a count of the lines changed, i mean
15:30 mrpeters senator: aha, ok.  i was thinking the ++++ and ---- were the number of insertions and deletions
15:30 tsbere mrpeters: I think those represent blocks of insertions and deletions, each of which is one or more lines long
15:30 mrpeters ok, cool.  what senator said makes more sense than what i was supposing
15:31 senator yes, one + means one hunk in which at least some lines are added, one - means one hunk is which at least some lines are removed
15:31 jeff they are a representation of a total. points on a graph.
15:31 mrpeters just trying to figure out what files we need to touch for deb building
15:32 mrpeters likely will just be the make_release and Makefile.install stuff
15:34 senator well, what i said about hunks isn't exactly correct, but you get the basic idea
15:36 mrpeters totally, thank you
15:36 mrpeters i coulda googled but i  knew id get a better response from a human :)
15:39 jeff man diffstat will give you some additional information.
15:39 jeff i don't know offhand if git calls out to diffstat or re-implements diffstat.
15:46 Bmagic joined #evergreen
15:56 stevenyvr joined #evergreen
16:01 smyers joined #evergreen
16:05 tsbere Anyone have a clue how to debug "Syndetics decided it can't give us images, but it is giving us other added content just fine" for a single bib? Note that we are, in fact, getting error documents instead of images.
16:09 jeff tsbere: what isbn/upc?
16:09 tsbere jeff: Er, three different ones on the record, actually.
16:10 jeff url to record?
16:10 tsbere http://catalog.mvlc.org/eg/opac/record/1022048
16:10 jeff thanks
16:13 Dyrcona Eh. Whatever. Syndetics wasn't so hot when we were on Horizon either.
16:13 bshum Fwiw, I've noticed this similar issue with our system too.  I just haven't had time to nail it down either.
16:17 bshum I feel like I was worried about stuff like parenthesis issues
16:17 Dyrcona GIGO
16:17 bshum Like it was ISBN (followed by stuff) in all the examples I saw broken
16:17 bshum But I don't remember
16:19 jeff hrm. probably not just a matter of commit cfa27b7
16:19 pinesol_green [evergreen|Jeff Godin] Fix detection of empty Syndetics AC data elements - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=cfa27b7>
16:19 jeff i wonder if data_exists is used for covers at all. maybe not.
16:20 Dyrcona Based on the date of the commit, we have that.
16:20 Dyrcona This record has ISBN (followed by stuff).
16:21 Dyrcona Why videodiscs are given ISBNS.... Nah, I don't care to know.
16:21 dbs ebook-on-videodisc I'm sure
16:22 jeff tsbere: can you get me the url from the logs for that record? grep -E '0131500449|0131500430|0130267570' logfile -- privmsg to protect your client id, or strip the client id...
16:22 jeff well, you could probably include the string http://syndetics.com/index.aspx?isbn= in your pattern as well, with proper escaping, etc, etc.
16:34 jeff tsbere: thanks for the url data -- i have a suspicion that it's sending http://syndetics.com/index.aspx?isbn=/sc.​gif&amp;upc=&amp;issn=&amp;client=DELETED to syndetics, which it shouldn't be doing in the first place, but we should also probably handle the error output better.
16:34 jeff tsbere: checking something here.
16:35 * tsbere is trying to find the proper part of the logs to see if he can spot something
16:36 jeff i suspect you'll find some lines matching this near where you have those containing those isbns: http://syndetics.com/index.aspx?isbn=/​mc.gif&amp;upc=&amp;issn=&amp;client=
16:36 jeff bshum++ for some additional data
16:36 jeff (confirming my suspicion :-)
16:37 bshum Yeah, took me a moment or two to finally get a quiet server to look for it.
16:37 jeff yeah, i can fix this. it's probably my bug. :-)
16:38 tsbere I can find lines like that. And lots of ones that are isbn=/??.gif&upc=VALUE&issn=&client=DELETED
16:39 jeff sure. those should be good, though. that's AV content with a UPC but no ISBN
16:39 jeff many of those will be placeholder nothing-found images, but some will be real jacket/cover images that you'd never have had before :-)
16:42 pinesol_green [evergreen|Dan Wells] Translation updates - po files - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=02a40d1>
16:42 pinesol_green [evergreen|Dan Wells] Translation updates - newpot - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=3f639a8>
16:45 jeff got it.
16:47 * bshum cheers jeff on
16:54 jeff yeah, okay. i need to get something else done here, but it comes down to Business::ISBN::_common_format not being able to handle '0131500449 (videodisc 2A)' even though it can handle things like '0131500449 (foo)' -- i'll create a bug with some examples of works / doesn't work and throw together a fix.
16:54 jeff essentially, pre-clean the isbns based on how we've done it elsewhere / in the past.
16:55 jeff and at the same time skip sending the query upstream if we have no identifiers, and "break less" if and when we get an error like this.
16:56 stevenyvr joined #evergreen
16:57 jeff and maybe fix up Business::ISBN, though it might simply be that it was never intended to handle what we're throwing at it.
16:58 bshum Aha
16:58 bshum And aww
17:04 smyers_ joined #evergreen
17:05 bshum jeff++
17:15 mmorgan left #evergreen
17:17 dcook joined #evergreen
17:21 smyers__ joined #evergreen
17:28 kmlussier It may be a little late in the day to ask an acq question, but I'll throw it out there anyway. We've been having a lot of discussions regarding the issue in bug 1270289
17:28 pinesol_green Launchpad bug 1270289 in Evergreen "Backordered items have line item status of cancelled" (affected: 3, heat: 14) [Undecided,Confirmed] https://launchpad.net/bugs/1270289
17:28 kmlussier And, as much as we talk about improving the display for backordered items, staff just havei a hard time accepting the fact that a canceled status is used at all for backordered items.
17:29 kmlussier I'm just curious if the decision to use a canceled status has anything to do with conforming with the EDI standard.
17:29 kmlussier IOW, is it something that can be changed or does it need to stay that way on the backend to make EDI happy.
17:48 berick kmlussier: it's not needed for EDI.  it just streamlined some back-end processing to have them share the status.  the bigger problem, IMO, is that we are showing the bare status strings.
17:48 berick that was not really planned, it just happened
17:49 berick they are meant only for computer consumption
17:49 jtaylorats joined #evergreen
17:49 berick and are not translate-able today
17:49 berick we should be showing human friendly status information instead, which knows the difference between really canceled and delayed
17:50 kmlussier Oh really? I didn't realize there was a translation issue there.
17:50 kmlussier I agree on the human friendly status information. Even without translation issues, I find backorder quantity to be an awkward phrase. :)
17:51 berick yeah, the UI just shows the value of lineitem.status right out of the database, which is not at translatable column
17:51 berick yeah, well, you can change the cancel-reason labels
17:51 berick those are editable / translatable
17:54 kmlussier Good to know. I'll pass that information along.
17:57 berick thinking we need to hide the bare/untranslated lineitem status string, show translatable ones instead for non-canceled lineitems, and show the acq.cancel_reason.label for canceled lineitems.
17:57 berick or something along those lines
18:08 kmlussier I'm also floating the idea of possibly using a delayed status for those that aren't truly canceled. I'm not sure what kind of feedback I'll get though.
18:09 kmlussier What I'm hearing is that some people want a true backordered status and not use a canceled status at all for those items. But I'm concerned about the scope of that project.
18:10 berick for the UI bits, that can all be done with some labeling.  i don't see any need really change any statuses, unless I'm missing something.
18:11 berick kmlussier: are you just taking about the PO/Picklist page.. and I guess the PO print template
18:11 kmlussier Yeah, that's what I'm shooting for. Making a clear distinction in the interface.
18:11 kmlussier And yes, I think we're just talking about the PO/Picklist page, but I've asked for examples in case I'm missing something.
18:11 berick ok, well that's certainly a smaller project than changing the actual lineitem.status values
18:12 berick which, again, should not be visible anyway
18:12 kmlussier OK, thanks for the feedback. It's very helpful.
18:13 * kmlussier runs off to eat dinner.
18:13 kmlussier But first...
18:13 kmlussier berick++
18:14 berick right back atcha
18:14 berick kmlussier++ # wranglin
18:22 mrpeters left #evergreen
18:27 smyers_ joined #evergreen
19:08 dMiller_ joined #evergreen
19:58 phasefx joined #evergreen
20:23 smyers__ joined #evergreen
22:55 jeff tsbere++ bshum++ for data useful in troubleshooting AC failures
22:58 jeff dbs++ keeping us on release notes
22:58 jeff dbwells++ 2.6 alpha
22:58 jeff (karma all the people)
22:58 jeff :P
23:04 phasefx_ joined #evergreen
23:06 mtate joined #evergreen
23:21 dbs jeff++
23:22 jeff heh. i ALMOST mentioned bug 1182253 as being the only bug with a "needsreleasenote" tag
23:22 pinesol_green Launchpad bug 1182253 in Evergreen "Add --since option to marc_export cl script" (affected: 1, heat: 6) [Wishlist,Incomplete] https://launchpad.net/bugs/1182253
23:23 jeff is that how it came to your attention?
23:26 rsinger joined #evergreen
23:26 dbs heh, no; I just ordered bugs with pullrequest by age and it was near the top of the list (ARRAY_ACCUM bug being slightly older)
23:44 jeff added release notes to bug 1246853 :-)
23:44 pinesol_green Launchpad bug 1246853 in Evergreen "Add button in user editor to update expire date" (affected: 1, heat: 6) [Wishlist,In progress] https://launchpad.net/bugs/1246853
23:44 jeff "it's such an obvious thing, and the commit message says it all" is an easy trap.
23:45 jeff though I'll admit I didn't use that thinking to dismiss the idea of adding a release notes entry, I simply didn't think to add one.

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat