| Time |
Nick |
Message |
| 05:00 |
pinesol_green |
News from qatests: Test Success <http://testing.evergreen-ils.org/~live> |
| 07:03 |
|
jvwoolf joined #evergreen |
| 07:14 |
|
JBoyer joined #evergreen |
| 07:19 |
|
book` joined #evergreen |
| 07:23 |
|
agoben joined #evergreen |
| 08:05 |
|
collum joined #evergreen |
| 08:43 |
|
collum_ joined #evergreen |
| 08:51 |
|
mmorgan joined #evergreen |
| 08:59 |
|
jeff___ joined #evergreen |
| 08:59 |
|
yboston joined #evergreen |
| 09:02 |
|
bos20k joined #evergreen |
| 09:11 |
|
JBoyer joined #evergreen |
| 09:35 |
mmorgan |
If I go ahead and purge a patron that has overdue and/or lost item charges, will anything break? |
| 09:36 |
mmorgan |
I can see that the open transactions get aged, and stay open, not sure that's a problem. |
| 09:41 |
* csharp |
sings "This testbox was made for breakin'... and that's what I will do..." |
| 09:42 |
mmorgan |
The Lost items stay Lost. If I check one in, I get an error, but I can just delete the Lost item, so it can't get checken in. |
| 09:43 |
mmorgan |
:) |
| 09:46 |
mmorgan |
We're planning on deleting a number of long inactive patron records, some of which have open transactions. Is it best to resolve the open transactions before deleting, or do we not really need to worry about it if we're not expecting the money or the materials back? |
| 09:50 |
JBoyer |
mmorgan, My assumption (keep that in mind) is that it won't really be an issue, especially if you pair it with a cleanup of items that no one ever expects to see again. (These sound like ideal candidates for such a list.) |
| 09:52 |
JBoyer |
We're preparing to start a cleanup of items in "bad" statuses that haven't changed in 5 years. |
| 09:53 |
mmorgan |
JBoyer: We're looking at that also. |
| 09:53 |
|
Dyrcona joined #evergreen |
| 09:55 |
mmorgan |
I guess my one concern with the Lost items is that the billed amount would at least need to be reported to the item's owning library. |
| 09:57 |
JBoyer |
mmorgan, That's definitely a good idea. Lost materials bills are one reason I don't think we've really bothered with old patron cleanup unless they have 0 bills. It takes a board resolution to be able to waive those here and most don't want to give up on those kinds of amounts. |
| 09:59 |
Dyrcona |
And George Washington still owes the NY Public Library, but I don't think they're seeing those items or the money.... :) |
| 10:02 |
JBoyer |
True, but in the back of some people's minds, it's still "letting people get away with" something. The eyes roll so hard. |
| 10:03 |
Dyrcona |
True, true.... |
| 10:03 |
Dyrcona |
I'll read the logs to see what I missed. I almost forgot about IRC this morning. :) |
| 10:05 |
Dyrcona |
When we purge patrons, we deliberately skip those with lost items. |
| 10:06 |
Dyrcona |
Evergreen doesn't like aged circulations with lost items. You get an error as you mentioned earlier. |
| 10:06 |
mmorgan |
Dyrcona: So they just remain indefinitely, along with the patrons? |
| 10:06 |
mmorgan |
Until the library decides to resolve it somehow? |
| 10:06 |
Dyrcona |
Generally, too, the libraries don't want us purging patrons with lost items for the reason JBoyer specified. |
| 10:06 |
Dyrcona |
mmorgan: Yes. |
| 10:07 |
mmorgan |
Yeah, but at some point, you gotta let go :) |
| 10:08 |
|
rlefaive_ joined #evergreen |
| 10:08 |
Dyrcona |
Um.. cf. George Washington and the NYPL. :) |
| 10:09 |
Dyrcona |
I even get crazy requests like don't purge if they owe more than X amount in fines, not just lost books. |
| 10:11 |
mmorgan |
Well, it can be difficult to write off the $$, especially if the money goes to the municipality. |
| 10:11 |
Dyrcona |
Yes. |
| 10:12 |
Dyrcona |
Looking at some recent scripts another criteria seems to be to keep if they have things checked out, so that covers lost items, too. |
| 10:13 |
mmorgan |
So, keep if a linked xact has no checkin_time? |
| 10:14 |
Dyrcona |
Yeah. Two academics asked for that recently when getting rid of expired patron accounts. |
| 10:15 |
mmorgan |
Oh, AND no xact_finish, in case they paid for a lost item. |
| 10:16 |
Dyrcona |
Right. |
| 10:16 |
Dyrcona |
I did a left join on circulation with certain criteria and did a circulation is null check. |
| 10:19 |
Dyrcona |
I used checkin_scan_time and xact_finish being null. |
| 10:21 |
Dyrcona |
That basically covers anyone with something checked out, which theoretically covers anything lost. |
| 10:25 |
mmorgan |
That clarifies things a bit :) |
| 10:25 |
mmorgan |
JBoyer++ |
| 10:25 |
mmorgan |
Dyrcona++ |
| 10:27 |
mmorgan |
Now, if I can only make sure that all the transactions that should be closed actually ARE closed. :-/ |
| 10:27 |
Dyrcona |
Hmm. I had a transaction closer script. |
| 10:28 |
Dyrcona |
Let me see if I can dig it out. I used it at MVLC before a bug was fixed. |
| 10:28 |
miker |
berick: if you're around and have a minute, would you mind weighing in on the last 2 comments of bug 1642086 ? |
| 10:28 |
pinesol_green |
Launchpad bug 1642086 in Evergreen "jQuery support for the public catalog" [Wishlist,New] https://launchpad.net/bugs/1642086 - Assigned to Mike Rylander (mrylander) |
| 10:28 |
berick |
miker: funny you should ask.. |
| 10:28 |
mmorgan |
Dyrcona: That would be most useful, thanks! |
| 10:28 |
miker |
:) |
| 10:31 |
* mmorgan |
runs off to a meeting. |
| 10:31 |
|
mmorgan1 joined #evergreen |
| 10:39 |
Dyrcona |
@later tell mmorgan Here's the script: https://github.com/Dyrcona/evergreen_utilities/blob/master/perl/xact_finisher.pl . Turns out, it isn't as smart as I thought it was. |
| 10:39 |
pinesol_green |
Dyrcona: The operation succeeded. |
| 10:40 |
Dyrcona |
Looks like it would blow up with recent Perl, too: "use UNIVERSAL;" |
| 10:50 |
Dyrcona |
Weird. I thought use UNIVERSAL was fatal, but apparently not. |
| 11:11 |
Bmagic |
Anything special I need to know to trim down action.circulation? Just DELETE FROM action.circulation where xact_finished is not null and xact_start < now()-'2 years'::interval ? |
| 11:16 |
Dyrcona |
Bmagic: There is a purge_circulations db function that you can use. |
| 11:17 |
Bmagic |
oh good! |
| 11:17 |
Dyrcona |
Maybe it's age_circulations... |
| 11:17 |
|
rlefaive joined #evergreen |
| 11:17 |
Bmagic |
oh wow, that function is much more complex |
| 11:18 |
Bmagic |
I bet it's better than executing a delete query |
| 11:18 |
Dyrcona |
Yes. |
| 11:18 |
Dyrcona |
It's slow depending on your settings. |
| 11:18 |
Dyrcona |
47277 | 52 days 23:17:39.657392 | 127.0.0.1 | psql | select * from action.purge_circulations(); |
| 11:18 |
Dyrcona |
In my training database. :) |
| 11:18 |
Bmagic |
I see, I need to change some settings in config.global_flag |
| 11:19 |
Dyrcona |
Telling it to delete things older than 3 years. |
| 11:19 |
Dyrcona |
Or something like that. |
| 11:19 |
Bmagic |
oh man, 52 days of run time? |
| 11:19 |
Dyrcona |
Yes. |
| 11:19 |
Bmagic |
holy cow. Maybe I need to set the setting to 10 years, and work my way up to 3 |
| 11:20 |
Dyrcona |
berick says they made a modified version to take a start and end date and ran that with different dates over a number of days. |
| 11:20 |
Bmagic |
right, that is my train of thought |
| 11:21 |
Bmagic |
we have over 8 million rows in action.circulation. It might be time to run this |
| 11:21 |
Dyrcona |
Doing a couple of months at a time shouldn't be too bad, but I suppose it depends on how you change the function. |
| 11:21 |
Dyrcona |
We have over 46 million. :) |
| 11:21 |
Bmagic |
that is a lot |
| 11:21 |
Dyrcona |
For certain definitions of a lot, yes. :) |
| 11:21 |
Bmagic |
my goal is to get the disk footprint smaller |
| 11:22 |
Dyrcona |
This won't do that. |
| 11:22 |
Bmagic |
and* make the database run "faster" |
| 11:22 |
Dyrcona |
It moves the circulations to aged_circulation which takes up more space per row, AFAICT. |
| 11:22 |
Bmagic |
You don't think moving action.circulation rows into action.aged_circulation doesn't trim the disk? |
| 11:22 |
Bmagic |
less indexes? |
| 11:22 |
Dyrcona |
How could it? It adds data. |
| 11:23 |
Dyrcona |
Well, smaller indexes might make the foot print smaller, I guess it depends on the difference. |
| 11:24 |
Bmagic |
aged_circulation has some indexes but it has (fewer?) than action.circulation ? |
| 11:24 |
Dyrcona |
Only way to find out is to check the db size before and after running it. |
| 11:24 |
Bmagic |
right on, well, sounds like fun |
| 11:24 |
Dyrcona |
BTW, at noon, it will be 53 days of run time. ;) |
| 11:26 |
Bmagic |
Do you run vacuum on the db with some kind of cron? |
| 11:26 |
berick |
Dyrcona: heh, i'm surprised you're still letting run. are you going to run it in prod that long? or are you just curious at this point how long it will take? |
| 11:27 |
Dyrcona |
Bmagic: Autovacuum happens on its own. |
| 11:27 |
Dyrcona |
berick: Partly the latter. A week or two ago we had a discussion about stopping it, but I decided we should let it run. |
| 11:28 |
* csharp |
doesn't depend on autovac alone and finds it useful to have a weekly vacuum analyze cron |
| 11:28 |
Bmagic |
there is no circumstance when we want to run vacuum? |
| 11:28 |
csharp |
we've hit xact wraparound warnings before - very un-fun |
| 11:28 |
Dyrcona |
Part of the argument was, that it had been running for over 40 days. If we stopped it, that would be 40 days we'd be behind. |
| 11:29 |
Dyrcona |
xact.id wrap around warnings? |
| 11:29 |
berick |
Bmagic: we've run a few VACUUM FULL's in cases where 90% of the rows were deleted, e.g. when we started aging action.hold_request and when we deleted several hundred million rows from auditor.asset_copy_history. |
| 11:29 |
Bmagic |
csharp: how do you solve that issue? Vacuum doesn't fix that does it? Increase the column type size? bigint? |
| 11:30 |
Bmagic |
oh yeah! auditor.* - is there a function to delete rows from tables in that schema?> |
| 11:30 |
miker |
Bmagic: once a table is big enough to need an index, "speed" is just a matter of selectivity of the query, it really doesn't have much to do with data size per se, only whether you ask it to find and return lots of it. IOW, if you are doing lookups by id (or similarly selective columns, like user), the size of the table is immaterial barring exceptional situations like what berick mentions (that won't be the case for the circulation table) |
| 11:30 |
csharp |
Bmagic: it's fixed with a vacuum analyze which reclaims the dead rows |
| 11:30 |
csharp |
and we do occasional vacuum full's with particular tables (like when check_postgres yells about bloat) |
| 11:31 |
Bmagic |
csharp: oh, so the issue isn't that it's running out of numbers in the sequence? |
| 11:31 |
miker |
csharp: sounds like you may want to tune autovac (art as much as science, of course) |
| 11:31 |
* miker |
runs to a meeting |
| 11:31 |
csharp |
miker: right |
| 11:31 |
berick |
and yes what csharp said, we've also hit the wraparound issues, particularly on action.hold_copy_map which has a lot of turn over |
| 11:31 |
csharp |
Bmagic: right, it's the internal-to-postgres xact IDs |
| 11:32 |
Bmagic |
I see, well, that is different. I suppose no one has run out of numbers in the sequence? Max 10 digits right? |
| 11:32 |
csharp |
Bmagic: not yet |
| 11:32 |
Bmagic |
9,999,999,999 |
| 11:33 |
Dyrcona |
I thought it went to 9 quintillion, not 9 billion.... |
| 11:33 |
Bmagic |
addressing auditor tables - just run DELETE WHERE ? |
| 11:34 |
csharp |
Bmagic: and I can't recommend check_postgres enough - both as a nagios/icinga/whatever check or as a standalone script to grab stats/info: https://github.com/bucardo/check_postgres |
| 11:34 |
Bmagic |
cool, I will definatly get into that |
| 11:34 |
Bmagic |
oh, we use zabbix |
| 11:34 |
Dyrcona |
Bmagic: We have a cron job to delete auditor data older than X days (where is a mulpiple of 30, likely less than or equal to 3, that I don't remember ottomh). |
| 11:35 |
berick |
Bmagic: i think a lot of people just 'delete from auditor.asset_copy_history where audit_time < now() - 'some interval'; i have a more nuanced script i can share if you're interested. it ensures that a certain number of entries are retained, for example. |
| 11:35 |
tsbere |
Bmagic: MVLC had a pile of DELETE FROM auditor.blah WHERE AGE(audit_time) > interval. Mostly '12 months' or '30 days'. |
| 11:36 |
Bmagic |
oh good, so, just delete queries, not calling a special function like action.circulation |
| 11:36 |
Dyrcona |
Right. Just delete. |
| 11:37 |
Dyrcona |
The circulation purge does a lot of extra stuff, and it gets called if you just delete a row. There's a trigger. |
| 11:37 |
Dyrcona |
So, you could just delete from action.circulation. |
| 11:49 |
Bmagic |
ok, nevermind on the max of bigint. It's extremely huge: 9,223,372,036,854,775,807 |
| 11:50 |
Bmagic |
2^63 |
| 11:51 |
Dyrcona |
Right, 9.2 quintillion. |
| 11:55 |
Bmagic |
I bet in our lifetime, we see that much data moving per second over our networks |
| 11:56 |
Dyrcona |
Maybe not... That's about half the estimated number of particles in the know universe, IIRC. |
| 11:57 |
Bmagic |
haha, I have to look this up |
| 11:57 |
Dyrcona |
Or maybe, 2^64 is... I know 2^128 is greater than the number of particles in the known universe. |
| 11:58 |
Bmagic |
Data is growing faster than ever before and by the year 2020, about 1.7 megabytes of new information will be created every second for every human being on the planet. |
| 11:58 |
Dyrcona |
Sure, sure...and we all know that data is a synonym for junk. :) |
| 11:59 |
Bmagic |
Facebook users send on average 31.25 million messages and view 2.77 million videos every minute. |
| 11:59 |
Dyrcona |
"Garbage in, garbage out," as the IRS used to say. |
| 11:59 |
Bmagic |
that term was coined by the IRS? |
| 11:59 |
Dyrcona |
Yes, I believe so. |
| 11:59 |
Dyrcona |
When they first started automating in the '60s. |
| 12:01 |
Bmagic |
https://www.forbes.com/sites/bernardmarr/2015/09/30/big-data-20-mind-boggling-facts-everyone-must-read/#160d5b2d17b1 |
| 12:01 |
|
_adb joined #evergreen |
| 12:05 |
|
khuckins_ joined #evergreen |
| 12:11 |
dbs |
Hmm, "Error retrieving stored z39.50 credentials" in XUL staff client after upgrading to 2.12 - sound familiar to anyone who searches Z39.50 servers that use authentication? |
| 12:13 |
tsbere |
I don't think we ever stored the credentials, just had staff enter them. |
| 12:17 |
csharp |
dbs: I've not heard of it... to be clear, this is in the client when entering the z39.50 UI and they've saved credentials to outside servers? |
| 12:18 |
dbs |
That's what it seems to be, yeah |
| 12:19 |
* dbs |
needs to dig further |
| 12:24 |
csharp |
dbs: I just added credentials on my 2.12 test server, then exited and re-opened the z39 UI and it worked as expected - I'll check with our catalogers who have been testing too |
| 12:24 |
Dyrcona |
Bmagic: Maybe my attribution to the IRS is another case of GIGO: http://www.atlasobscura.com/articles/is-this-the-first-time-anyone-printed-garbage-in-garbage-out |
| 12:26 |
Dyrcona |
And I love that Charles Babbage quote. I have read it before. |
| 12:27 |
csharp |
dbs: in our case it may not be a good test since most are creating new XUL client profiles as they log into our test server |
| 12:27 |
csharp |
so not exactly an "upgrade" |
| 12:29 |
Bmagic |
Dyrcona: nice article! |
| 12:33 |
dbs |
csharp: thanks! |
| 12:34 |
csharp |
dbs: confirmed from our lead cataloger that no one has reported similar problems while testing 2.12 |
| 12:34 |
csharp |
happy to help! |
| 12:34 |
dbs |
We only have one entry in config.z3950_source_credentials and it's mine, so it really looks like a XUL thing. I'll dig deeper. |
| 12:39 |
|
mmorgan joined #evergreen |
| 13:12 |
Dyrcona |
hm... too bad there are no good guidelines for random page cost. I'm tempted to set it to 1.0 on my new server. |
| 13:14 |
Dyrcona |
Make it the same as sequential page cost.... but I'll leave it at 2.0. |
| 13:16 |
|
abowling joined #evergreen |
| 13:21 |
Dyrcona |
Well, drop it to 2.0. "Leave it at 2.0" is a comparison to the current production db, just to be perfectly clear. |
| 13:25 |
berick |
Dyrcona: FYI. i'll open a bug soon. http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/berick/open-auth-login |
| 13:26 |
Dyrcona |
berick: Cool! |
| 13:26 |
Dyrcona |
That should be easy to test. |
| 13:27 |
Dyrcona |
We can start by altering the authenticate method of Cronscript and friends. |
| 13:29 |
berick |
yeah, and my initial tests look good. confirmed block count handling still works as expected. no mem leaks. |
| 13:31 |
Dyrcona |
Yeah, I was just looking at all of the "free me" comments. :) |
| 13:31 |
Dyrcona |
That's a good idea... |
| 13:35 |
|
jwoodard joined #evergreen |
| 13:42 |
|
jihpringle joined #evergreen |
| 15:03 |
|
mmorgan1 joined #evergreen |
| 15:06 |
|
_adb joined #evergreen |
| 15:08 |
|
RBecker joined #evergreen |
| 15:14 |
JBoyer |
Before I try to poke the LP search bear, does anyone have a # handy for a bug about being able to set an SMS number without a carrier? |
| 15:15 |
abneiman |
jboyer: bug 1098685 ? |
| 15:15 |
pinesol_green |
Launchpad bug 1098685 in Evergreen 2.12 "User can select SMS notify without providing a valid address" [Medium,Fix committed] https://launchpad.net/bugs/1098685 |
| 15:16 |
JBoyer |
YUP. |
| 15:16 |
JBoyer |
abneiman++ |
| 15:16 |
JBoyer |
Looks like we're about to upgrade to 2.12.5... |
| 15:16 |
JBoyer |
(ish) |
| 15:34 |
rjackson_isl |
it looks like they can still get the data out of sync: https://bugs.launchpad.net/evergreen/+bug/1098685/comments/19 ? |
| 15:34 |
pinesol_green |
Launchpad bug 1098685 in Evergreen 2.12 "User can select SMS notify without providing a valid address" [Medium,Fix committed] |
| 15:34 |
rjackson_isl |
oops - last comment was for Jboyer (Duh for the day - among others...) |
| 15:34 |
|
jeff_ joined #evergreen |
| 16:02 |
|
khuckins__ joined #evergreen |
| 16:06 |
|
jeff_ joined #evergreen |
| 16:17 |
|
mmorgan joined #evergreen |
| 16:22 |
|
jvwoolf joined #evergreen |
| 16:25 |
|
b_bonner left #evergreen |
| 16:26 |
|
b_bonner joined #evergreen |
| 16:26 |
|
StomproJ joined #evergreen |
| 16:31 |
|
Jillianne joined #evergreen |
| 16:47 |
|
khuckins_ joined #evergreen |
| 16:53 |
bshum |
gmcharlt: Just ran a new POT sync and put the commit in http://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/bshum/20180815-i18n-pot-sync |
| 16:53 |
bshum |
I was reading through what changed for webstaff though |
| 16:53 |
bshum |
And feel like maybe there's some potential bugs there in the strings |
| 16:54 |
bshum |
For example: |
| 16:54 |
bshum |
#: ../../Open-ILS/src/templates/staff/circ/share/t_bad_barcode_dialog.tt2:10 |
| 16:54 |
bshum |
msgid "" |
| 16:54 |
bshum |
"\n" |
| 16:54 |
bshum |
-"Bad check digit, possibly due to a bad scan. Use this barcode (\"%1\") anyway?\n" |
| 16:54 |
bshum |
+"Invalid barcode: <div class=\"strong-text-4\">%1</div>\n" |
| 16:54 |
bshum |
+"Your entry has a bad check digit, possibly due to a bad scan.<br/>\n" |
| 16:54 |
bshum |
+"Choose Cancel to try again, or Accept to use barcode <b>%1</b> anyway.\n" |
| 16:54 |
bshum |
That looks like we added some div stuff inside the translation code |
| 16:54 |
bshum |
Which doesn't seem good |
| 16:54 |
* bshum |
should have pasted that, sorry |
| 16:55 |
bshum |
I'll read through more of it later, later tonight |
| 16:55 |
gmcharlt |
looking at the original string, the divs were there |
| 16:58 |
gmcharlt |
yeah, for that one it appears to be correclty parsing the [% |l(args) %]string[% END %] form |
| 17:01 |
pinesol_green |
News from qatests: Test Success <http://testing.evergreen-ils.org/~live> |
| 17:06 |
|
mmorgan left #evergreen |
| 17:07 |
|
jboyer-isl joined #evergreen |
| 17:16 |
bshum |
gmcharlt: Gotcha, then I won't worry about that too much |
| 17:16 |
bshum |
The only other one that I wasn't sure on was this change: |
| 17:16 |
bshum |
+msgid "<NONE>" |
| 17:16 |
bshum |
+msgstr "" |
| 17:16 |
bshum |
-msgid "<NONE>" |
| 17:16 |
bshum |
-msgstr "" |
| 17:16 |
bshum |
I wasn't sure if that was something local on my system side in how it was reading the symbol maybe |
| 17:18 |
gmcharlt |
bshum: that looks OK; was intentionally changed per commit d6fba7b4 so that "<NONE>" would actually display in the selctors |
| 17:18 |
pinesol_green |
gmcharlt: [evergreen|Jason Etheridge] lp1686459 webstaff: item editor form validation - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=d6fba7b> |
| 17:19 |
bshum |
Okay, then I think the POT changes are safe to commit to master then |
| 17:19 |
gmcharlt |
shall I just go ahead and do that now? |
| 17:21 |
bshum |
Sure |
| 17:22 |
bshum |
Yes please :) |
| 17:22 |
gmcharlt |
boom |
| 17:26 |
pinesol_green |
[evergreen|Ben Shum] Translation updates - newpot - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=b17fad6> |
| 17:35 |
Bmagic |
Weird, my primary postgres database decided to restart by itself..... |
| 17:38 |
Bmagic |
postgres[26808]: [3-1] 2017-08-15 16:04:30 CDT LOG: received fast shutdown request |
| 18:19 |
|
jvwoolf joined #evergreen |
| 18:19 |
|
jvwoolf left #evergreen |
| 20:12 |
|
_adb joined #evergreen |
| 20:12 |
|
_bott_ joined #evergreen |
| 20:16 |
|
dkyle joined #evergreen |
