| Time |
Nick |
Message |
| 07:43 |
|
graced joined #evergreen |
| 07:48 |
|
sarabee joined #evergreen |
| 07:53 |
|
ericar joined #evergreen |
| 07:53 |
|
graced joined #evergreen |
| 07:55 |
|
jboyer-isl joined #evergreen |
| 08:00 |
|
Newziky joined #evergreen |
| 08:02 |
|
rjackson_isl joined #evergreen |
| 08:11 |
|
collum joined #evergreen |
| 08:19 |
csharp |
eeevil: the explain analyze is just giving me "Function Scan on query_parser_fts (cost=0.25..10.25 rows=1000 width=64) (actual time=23179.184..23179.185 rows=2 loops=1)" - I'm assuming that's not what you're after, right ;-) |
| 08:20 |
csharp |
? |
| 08:20 |
csharp |
eeevil: just wondering what the best way to get to the core query is |
| 08:20 |
eeevil |
csharp: nope :) ... pull the SELECT that's passed to the function -- that's the core query |
| 08:20 |
|
Shae joined #evergreen |
| 08:20 |
csharp |
ok |
| 08:25 |
csharp |
eeevil: sorry - I'm not getting it :-/ - here is the full query inside the function: http://pastie.org/10062550 - where does the query I need to explain start and end? |
| 08:26 |
eeevil |
the query is bounded by the string: $core_query_11190$ |
| 08:26 |
csharp |
okay - good |
| 08:28 |
|
akilsdonk joined #evergreen |
| 08:32 |
csharp |
eeevil: got it - I attached the explain analyze output and the core query to bug 1438136 |
| 08:32 |
pinesol_green |
Launchpad bug 1438136 in Evergreen "OPAC searching significantly slowed by adding format filters" (affected: 1, heat: 6) [Undecided,New] https://launchpad.net/bugs/1438136 |
| 08:36 |
|
mmorgan joined #evergreen |
| 08:41 |
|
pmurray joined #evergreen |
| 08:56 |
jboyer-isl |
csharp: I assume this search includes the format filters, it might be useful to see how the non-filtered search is different to compare. |
| 09:07 |
eeevil |
csharp: yep: Bitmap Index Scan on mrca_vlist_idx (cost=0.00..337.66 rows=1821 width=0) (actual time=515.026..515.026 rows=1587948 loops=1) ... 1.8k est rows vs 1.6M actual rows. first thing I'd suggest is remove the duplicate condition in the vlist clause (change '(61|311)&(61|311)' to '61|311' |
| 09:09 |
eeevil |
csharp: if that changes the plan, then there's a code change that could help (dedup vlist searches before passing to PG). if not, you'll want to increase the stats target on record_attr_vlist and vac-analyze |
| 09:20 |
|
Dyrcona joined #evergreen |
| 09:27 |
|
Newziky1 joined #evergreen |
| 09:32 |
|
dkyle1 joined #evergreen |
| 09:32 |
|
maryj joined #evergreen |
| 09:44 |
|
yboston joined #evergreen |
| 10:00 |
Dyrcona |
@quote random |
| 10:00 |
pinesol_green |
Dyrcona: Quote #52: "<dbs> MARC is not machine readable." (added by Dyrcona at 02:34 PM, April 12, 2013) |
| 10:08 |
jonadab |
To the casual observer, MARC looks like an old-timey 8-bit binary format got transmitted over a 7-bit medium. |
| 10:12 |
|
mrpeters joined #evergreen |
| 10:30 |
|
mllewellyn joined #evergreen |
| 10:39 |
bshum |
gmcharlt: Our reports blew up about six days ago (with a bad report template that went to get the circ notes for everything with a circ mod "book" in the consortium) |
| 10:39 |
bshum |
So I'm taking this reset time to try out the clipping of Clark's cape |
| 10:39 |
gmcharlt |
bshum: groovy |
| 10:40 |
bshum |
I'm trying to decide if 1000 minutes might be excessive yet (that's like 16+ hours) |
| 10:40 |
bshum |
I'll have to ask around the office to see what seems like a nice stop point for us. |
| 10:41 |
bshum |
Wait, 60 |
| 10:41 |
bshum |
I misread the line didn't I? |
| 10:41 |
* bshum |
has his eyes checked |
| 10:42 |
bshum |
gmcharlt: I'll let you know what comes from our testing and maybe this might be one of the first new features we push on next |
| 10:43 |
gmcharlt |
bshum: in addition to asking folks -- http://paste.lisp.org/display/146642 |
| 10:44 |
bshum |
gmcharlt++ # I like that better :) |
| 10:45 |
gmcharlt |
bshum: data from that applied to our hosted customers is what led me to setting 60 minutes as a default |
| 10:46 |
bshum |
So with that, I see less than eleven reports historically taking longer than 60 minutes. |
| 10:46 |
bshum |
Most of them have been done in 5 minutes or less. |
| 10:46 |
gmcharlt |
obvious, YMMV, but arguably any report that finishes but runs longer than 60 minutes probably could have been better done using hand-crafted SQL anyway |
| 10:48 |
jeff |
bshum: do you have access logs sufficient to determine if those >60m reports resulted in output that was actually viewed? :-) |
| 10:48 |
bshum |
jeff: Yeah I doubt we do :) |
| 10:48 |
|
sandbergja joined #evergreen |
| 10:56 |
|
Shae_ joined #evergreen |
| 10:58 |
Dyrcona |
I have rarely seen a report take longer than 15 minutes to run, and most run in just 1 to 2 minutes or less. |
| 10:58 |
Dyrcona |
The few times that I have looked. |
| 10:59 |
bshum |
gmcharlt: Hmm, getting errors like http://pastie.org/10062847 while reports attempt to run. |
| 10:59 |
bshum |
that's what spills out onto the console anyways |
| 11:02 |
gmcharlt |
bshum: hmm, have you used either the CLI switch or opensrf.xml to explicitly set a limit for the size of the resultset? |
| 11:02 |
bshum |
I added the new values to opensrf.xml |
| 11:03 |
bshum |
But let me check to make sure it saved right |
| 11:03 |
bshum |
Aha |
| 11:03 |
bshum |
It's empty |
| 11:03 |
bshum |
It was added, but there's no value there |
| 11:04 |
bshum |
So the sample entry was <resultset_limit></resultset_limit> |
| 11:05 |
gmcharlt |
hmm, and getting passed back by opensrf.settings as a hashref containing undef, perhaps |
| 11:07 |
bshum |
Hmm, 1 million rows? :D |
| 11:11 |
|
vlewis joined #evergreen |
| 11:17 |
Dyrcona |
One meellyun rows of what? |
| 11:18 |
bshum |
Just setting some unreasonable number of rows to test the new clark out. |
| 11:18 |
bshum |
Should probably set it lower to really test it. |
| 11:18 |
gmcharlt |
:) |
| 11:18 |
bshum |
But eh, I'll do that on some other server. |
| 11:18 |
bshum |
And let's say, not production :D |
| 11:21 |
kmlussier |
bshum: Where's your sense of adventure? Testing in production is fun! :D |
| 11:22 |
Dyrcona |
The only real testing is done in production when real users get their hands on things. ;) |
| 11:25 |
bshum |
Oh duh |
| 11:26 |
bshum |
I have to restart services for the new limitset to apply :) |
| 11:26 |
bshum |
I was wondering why it was still broken |
| 11:26 |
Dyrcona |
Well, settings and Clark would probably need to be restarted. |
| 11:26 |
bshum |
Yep |
| 11:26 |
Dyrcona |
Not sure it affects anything else. |
| 11:29 |
|
dreuther joined #evergreen |
| 11:30 |
bshum |
Yep, after restarting things, we're back to happily running reports. |
| 11:30 |
bshum |
I'll update the bug with notes about the potential issue with resultset_limit |
| 11:30 |
bshum |
And I'll add anything else that we find as we start using it. |
| 11:40 |
jboyer-isl |
bshum++ # testing! |
| 11:41 |
jboyer-isl |
gmcharlt++ # I ran that query and we had a 2.5 day report a while back. D: at least it didn’t cause any issues as it slowly trod on. |
| 11:41 |
bshum |
We'll schedule an evening to test the report past 1 hour and see if it kills as advertised. |
| 11:47 |
csharp |
eeevil: thanks for the response - deduping didn't change the plan, so I'll experiment with the stats target |
| 11:58 |
csharp |
postgresql_docs++ |
| 11:58 |
csharp |
that is possibly the best-documented F/LOSS project I've come across |
| 11:59 |
* csharp |
will limit his statement to thing he has actually used, though ;-) |
| 11:59 |
csharp |
s/thing/things/ |
| 11:59 |
Dyrcona |
csharp: Where the foot gun in that? ;) |
| 11:59 |
csharp |
Dyrcona: ha! |
| 12:02 |
gmcharlt |
bshum++ # edge case finder |
| 12:02 |
gmcharlt |
(and we ought to make a badge for that) |
| 12:02 |
|
jihpringle joined #evergreen |
| 12:03 |
jboyer-isl |
Game-ify all the things! |
| 12:12 |
Dyrcona |
And bshum's comment on the bug reminds me that the daemonize function from OpenSRF::Utils doesn't do a few things that it should. |
| 12:15 |
bshum |
Hmm |
| 12:17 |
gmcharlt |
Dyrcona: ... and serveral things that perhaps ought to be using O::Utils->daemonize() aren't, in favor of hand-coding |
| 12:17 |
gmcharlt |
whee! |
| 12:18 |
Dyrcona |
gmcharlt: I also wonder why we don't just use Proc::Daemon. |
| 12:19 |
gmcharlt |
Dyrcona: a shameful lack of TARDIS, perhaps :) |
| 12:20 |
gmcharlt |
but seriously, +1 to using something that does all the recommended bits... and specializes in using the recommended bits |
| 12:20 |
Dyrcona |
gmcharlt: Heh. I thought that was reason number 1. |
| 12:35 |
csharp |
eeevil: hmm - so I did 'alter table metabib.record_attr_vector_list alter COLUMN vlist set statistics 10000;', then vacuum analyzed, but I don't see a change in the plan yet and the query is still slow :-/ |
| 12:36 |
* csharp |
wonders if there's another table or column that needs that change too |
| 12:39 |
csharp |
I also tried de-duping after the statistics change too, in case that helps |
| 13:21 |
|
dreuther_ joined #evergreen |
| 13:54 |
bshum |
Hmm |
| 13:55 |
bshum |
So we have a patron with $16.00 in fines (consortium wide - $4.50 in home lib, the rest elsewhere) and they don't get a group penalty from the home library which blocks at $5.00. I thought it used to calculate penalty based on total owed, not just the amount owed to the lib with the penalty. |
| 13:56 |
bshum |
Or maybe something else is awry here... |
| 14:00 |
tsbere |
bshum: No, that sounds like how it works. It gets fun, though, when there are multiple layers. It won't update all layers. <_< |
| 14:00 |
bshum |
That's odd then. I guess we just never noticed it working that way, and always thought it was counting globally. |
| 14:00 |
bshum |
Good times! |
| 14:01 |
tsbere |
Nah. The way it counts is annoying at times, but I understand why it does what it does. If you want me to walk you through the pile of things I have discovered let me know. |
| 14:02 |
bshum |
I think I get it. |
| 14:02 |
bshum |
I'm just going to use this as a "so this is why we need a global fine penalty block instead of 80 different ones" unification, have one rule that actually works spiel. That'll go absolutely nowhere. |
| 14:08 |
|
ningalls joined #evergreen |
| 14:08 |
csharp |
@blame per-library policies |
| 14:08 |
pinesol_green |
csharp: per-library policies forgot to give the gerbils their chocolate-frosted sugar bombs |
| 14:08 |
csharp |
@blame search nice |
| 14:08 |
pinesol_green |
csharp: 1 found: #9: "$who is why we can never have nice things!" |
| 14:09 |
csharp |
@blame 9 per-library policies |
| 14:09 |
pinesol_green |
csharp: per-library policies is why we can never have nice things! |
| 14:09 |
bshum |
Heh |
| 14:23 |
|
Newziky joined #evergreen |
| 14:35 |
|
dreuther joined #evergreen |
| 14:37 |
|
DPearl joined #evergreen |
| 14:38 |
kmlussier |
In the existing client, staff need to click "Replace Barcode" to enter a new barcode for the user. In the web client. the "Replace Barcode" button doesn't do anything, but staff can just type in the new barcode. |
| 14:39 |
kmlussier |
I'm trying to figure out if it's a bad thing that they can just type in a new barcode now like they can with any other field on that record. |
| 14:39 |
* kmlussier |
assumes there was a reason we prevented staff from doing so in the old client. |
| 14:40 |
tsbere |
For a user replacing the barcode involves making the old one inactive and non-primary in addition to creating the new one. I don't know if the web client is doing that as well. |
| 14:40 |
bshum |
Ugh, that sounds "not good" |
| 14:41 |
jeff |
or "handy", depending on your workflow. ;-) |
| 14:45 |
DPearl |
bshum++ |
| 14:45 |
kmlussier |
Well, I included it on a broader bug report if anyone wants to offer opinions on how it should work. bug 1438358 |
| 14:45 |
pinesol_green |
Launchpad bug 1438358 in Evergreen "Web staff client: non-functional buttons on patron registration screen" (affected: 1, heat: 6) [Undecided,New] https://launchpad.net/bugs/1438358 |
| 14:51 |
Bmagic |
under what circumstance would the shelf_time be different than the capture_time? |
| 14:52 |
mmorgan |
Bmagic: when the item goes in transit to the pickup library. |
| 14:52 |
Bmagic |
lol |
| 14:52 |
Bmagic |
gotcha |
| 14:53 |
mmorgan |
We get a lot of that around here ;-) |
| 14:58 |
|
sfu joined #evergreen |
| 14:59 |
|
sfu left #evergreen |
| 15:24 |
|
Dyrcona joined #evergreen |
| 15:25 |
|
mglass joined #evergreen |
| 15:25 |
|
vlewis_ joined #evergreen |
| 16:11 |
|
dreuther_ joined #evergreen |
| 16:12 |
|
maryj joined #evergreen |
| 16:30 |
|
jboyer-isl left #evergreen |
| 16:39 |
|
dreuther joined #evergreen |
| 17:04 |
bshum |
kmlussier: I just tested https://bugs.launchpad.net/evergreen/+bug/1438410 and can confirm it doesn't work for my webclient. |
| 17:04 |
pinesol_green |
Launchpad bug 1438410 in Evergreen "Web staff client: Load patron from Checkout does not work" (affected: 2, heat: 10) [Low,Confirmed] |
| 17:04 |
bshum |
But it's fine for XUL, so far as I can tell. |
| 17:04 |
bshum |
I've updated the bug ticket accordingly. |
| 17:04 |
bshum |
But we might want to adjust the description of the problem a bit. |
| 17:04 |
kmlussier |
Done |
| 17:04 |
bshum |
kmlussier++ # testing |
| 17:05 |
|
mmorgan left #evergreen |
| 17:16 |
pinesol_green |
Incoming from qatests: Test Success - http://testing.evergreen-ils.org/~live/test.html <http://testing.evergreen-ils.org/~live/test.html> |
| 17:49 |
|
Newziky1 joined #evergreen |
| 23:15 |
|
gsams joined #evergreen |
| 23:42 |
|
akilsdonk joined #evergreen |
