| Time |
Nick |
Message |
| 00:37 |
|
Keith_isl joined #evergreen |
| 02:20 |
|
JBoyer joined #evergreen |
| 06:00 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 07:05 |
JBoyer |
Bmagic, that almost sounds like lp 1847805, with pcrud grabbing as many rows as you ask for then removing the ones you're not allowed to see, leaving you with less than you expected. |
| 07:05 |
pinesol |
Launchpad bug 1847805 in Evergreen "pcrud search can fail to retrieve rows that the user has access to" [High,Confirmed] https://launchpad.net/bugs/1847805 |
| 07:12 |
|
rjackson_isl_hom joined #evergreen |
| 07:26 |
|
rjackson_isl_hom joined #evergreen |
| 07:29 |
|
rjackson_isl_hom joined #evergreen |
| 08:13 |
|
mantis1 joined #evergreen |
| 08:30 |
|
collum joined #evergreen |
| 08:30 |
|
rfrasur joined #evergreen |
| 08:37 |
|
mmorgan joined #evergreen |
| 08:55 |
|
Dyrcona joined #evergreen |
| 09:02 |
|
jvwoolf joined #evergreen |
| 09:12 |
Bmagic |
JBoyer++ # thanks for the lead.I do see the query completing, then checking VIEW_HOLD perms a bunch of times. And then I keep seeing a query for +atevdef, and then that insane query in the paste above https://pastebin.com/PWWN5AbR |
| 09:14 |
Bmagic |
as best I can tell, it's gathering up all the opt_in_settings from any event definition. In this particular database, there are 149 action trigger defs with the exact same opt_in_setting. It takes that and makes a JSON query like what you see there |
| 09:17 |
JBoyer |
That settings query is busted, but I don't think it's causing your problem. You can see if its the pcrud problem by using a user that has VIEW_HOLD globally or a super user account and see if the numbers are always the total you expect. |
| 09:30 |
Bmagic |
naw, you're right, I'm on a less active machine now, and I don't think that is part of the issue. I think it was a red herring from another user on the system. |
| 09:30 |
Bmagic |
I have finally captured the server logs that produce the issue |
| 09:36 |
|
terranm joined #evergreen |
| 09:51 |
Bmagic |
The log looks fine. Other than, at the end it "rolling back db session" - Did it just take too long? |
| 09:52 |
Bmagic |
The initial query is 9 seconds, followed by a bunch of VIEW_HOLD lookups, slightly more than 10 seconds in total |
| 09:52 |
Dyrcona |
Bmagic: It could be, but rolling back db session happens a lot more than you would think. |
| 09:52 |
Dyrcona |
pcrud can make transactions when it isn't strictly necessary for various reasons. |
| 09:53 |
Bmagic |
right, I understand. It's the last thing it does. And there are no errors that I can see |
| 09:53 |
Dyrcona |
Did the query involve an update? |
| 09:54 |
Bmagic |
see if you can see anything off |
| 09:54 |
Bmagic |
https://pastebin.com/ZKxYbcmq |
| 09:55 |
Dyrcona |
TL;DR. :) |
| 09:55 |
Dyrcona |
I have a meeting in 5 minutes. |
| 09:55 |
Bmagic |
no problem, I'm sure I'll still be puzzling over it |
| 09:56 |
|
Keith-isl joined #evergreen |
| 09:57 |
Dyrcona |
I think you're hitting one of the client bugs involving too many repeated settings and permissions look ups. |
| 09:57 |
Dyrcona |
*poof* |
| 10:07 |
Bmagic |
I'm pretty sure that the issue is in the initial query somehow. The number of VIEW_HOLD perms lookups matches the number of results that I get in the client. The database is giving Perl a limited number of rows. |
| 10:19 |
Bmagic |
Now I'm really confused. I hunted down the resulting SQL query in the PG logs, and manually ran it, it gave me 25 rows! |
| 10:21 |
Dyrcona |
Bmagic: Is the authsession valid when the request is made? I seem to recall someone suggesting that an expired authsession could be a factor. |
| 10:22 |
Bmagic |
I just logged in before accessing the interface |
| 10:23 |
* Dyrcona |
poofs back to his Zoom meeting. |
| 11:00 |
Bmagic |
I keep coming back to time. I think* that there is a time limit somewhere in the stack, where if the drone takes longer than X... (that number sure seems to be 10 seconds), then it cuts bait and sends what it has. I can track down the query and run it, it returns the right number |
| 11:00 |
Bmagic |
But the interface truncates it to less than that number of rows, and turns off pagination |
| 11:06 |
miker |
Bmagic: jfyi, re your "marc to xml wrapper" function, I recommend looking at as_xml_record to simplify the post-processing: https://metacpan.org/pod/MARC::File::XML#as_xml_record(%5B$format%5D) (he said a day later) |
| 11:07 |
Bmagic |
miker++ # thanks, I'll take a close look when I get back to that project |
| 11:45 |
Dyrcona |
miker: Does your suggestion of using as_xml_record() make a practical difference in Evergreen? I'm asking because a) I'm not sure and b) I have a lot of code that uses as_xml, and if your suggestion is better, I'll gladly change my code. |
| 11:48 |
Dyrcona |
Well, most of my code uses OpenILS::Utils::Normalize::clean_marc which uses as_xml_record..... |
| 11:51 |
miker |
Dyrcona: Evergreen doesn't want the <collection> wrapper that as_xml() forces, so IMO "yes". as_xml_record() frees you from having to juggle the namespace attribute and that <wrapper> |
| 11:51 |
Dyrcona |
Don't mind me and my silly questions.....I just got out of a meeting and catching up on scrollback, and I'm not on top of my game today. |
| 11:51 |
miker |
:) |
| 11:51 |
|
jihpringle joined #evergreen |
| 11:51 |
Dyrcona |
miker: Yeah, I looked at the differences. I wasn't sure it was exactly meaningful, but it can be. |
| 11:52 |
Dyrcona |
Also, I just grepped my code and I use either clean_marc from Normalize.pm or I'm using as_xml_record, so ... what I said about not minding me and my quesitons. :) |
| 11:53 |
* Dyrcona |
tries to clear the brain fog. |
| 11:54 |
Dyrcona |
Maybe lunch will help. :) |
| 12:01 |
Bmagic |
BINGO! (I think) - this pull list thing. The PG query provides the right number of rows but the interface only shows some of them: Looking closely at the PG results, it's resulting in identical rows. Same ahr ID, same everything. |
| 12:05 |
Bmagic |
stripping out the wrapping, going down to the bare query from the fm_IDL.xml, I find that it still results in two rows for the hold ID |
| 12:06 |
Dyrcona |
Bmagic: Which IDL object? |
| 12:07 |
Bmagic |
ahopl |
| 12:09 |
Bmagic |
slowly but surely, I'm unravelling the mystery! I'm slowly commenting out some of these joins. It's returning one row now! after getting rid of LEFT JOIN config.standing_penalty |
| 12:11 |
Dyrcona |
So, to recap, you were seeing a bunch VIEW_HOLD perm look ups, and you're looking now at what the pull list does? I think you're on to something. If the web staff client uses pcrud. |
| 12:12 |
Bmagic |
The number of VIEW_HOLD lookups matched the number of rows in the interface. I asked for 10, it gave me 9, there were 9 VIEW_HOLD perm lookups... but tracking back to the PG query, there were 10 rows returned. Turns out, there are duplicate hold ID's in the return |
| 12:13 |
Dyrcona |
Yes. That makes sense given what you're saying now. |
| 12:13 |
Dyrcona |
I don't think the duplicate rows are the problem so much as the permission check for every row. |
| 12:14 |
Bmagic |
and now, I've figured out why. Some patrons have more than one penalty |
| 12:15 |
Bmagic |
The original problem is pagination is gone. And I think* that's automatically disabled when the interface doesn't get the number of rows that it asked for (it assumes that we're at the end of the results) |
| 12:15 |
Dyrcona |
OK. That also makes sense. |
| 12:16 |
Bmagic |
I don't know that for sure, just a hunch |
| 12:16 |
Bmagic |
I'm about to edit fm_IDL, throwing a DISTINCT in there |
| 12:16 |
Dyrcona |
A distinct on (ahr.id)? |
| 12:18 |
Bmagic |
there are a ton of columns |
| 12:18 |
Bmagic |
DISTINCT all of em |
| 12:19 |
Bmagic |
boom! |
| 12:19 |
Bmagic |
fixed it |
| 12:20 |
Bmagic |
OK, I think it's a bug. It only shows it's head when one of the patrons that has a copy on the pull list has more than one non-CAPTURE-blocking penalty |
| 12:21 |
Dyrcona |
Bmagic: I'm still not clear on what you're fixing, pagination not showing up? |
| 12:21 |
Bmagic |
I'm not sure that DISTINCT is the right solution. Perhaps a GROUP BY is better? But it would suffer from the column count changing in ahr.* |
| 12:22 |
Bmagic |
Yes, the interface essentially breaks when the number of rows that gets filled in from the server isn't equal to the number of rows we are expecting it to produce. You can't page next. Plus, it's confusing. You can as for 10 rows, and you will get 9. Then ask for 25 and you will get 15 |
| 12:23 |
Dyrcona |
OK. I'm not sure changing the IDL query is the right thing to do, either. That query is used by the backend and fixing it for the staff client pull list could break something else. |
| 12:24 |
Bmagic |
you mean there is code that expects it to produce duplicate rows? |
| 12:24 |
Bmagic |
the rows are 100% duplicate. Every column |
| 12:26 |
Dyrcona |
It's not displaying anything from config.standing_penalty? |
| 12:26 |
Bmagic |
nope |
| 12:26 |
Bmagic |
it just makes sure that they don't have a penalty that blocks CAPTURE |
| 12:26 |
Dyrcona |
I haven't looked at the actual code, yet. |
| 12:27 |
|
collum joined #evergreen |
| 12:27 |
|
mmorgan joined #evergreen |
| 12:27 |
Dyrcona |
OK. You can Lp this if you want and post a branch. Someone can always look at it to see if there's not a better way to resolve it. |
| 12:28 |
Dyrcona |
I might look later. I'm doing something else, and then I have a 1:00 PM meeting. |
| 12:28 |
Bmagic |
yeah, that's where I'm headed. Right now, I'm working on a query to produce a likely candidate pull list with the bug |
| 12:28 |
Bmagic |
Dyrcona++ # thanks for being my sounding wall |
| 12:28 |
Dyrcona |
Yeah. It would be good to add data in concerto that will reproduce the issue. |
| 12:29 |
Dyrcona |
Bmagic++ |
| 12:35 |
Bmagic |
see if this returns any results for anyone: https://pastebin.com/QpPgaBPX |
| 12:36 |
Bmagic |
if it does, then register a workstation at one of those branches, login, open the pull list |
| 12:45 |
Bmagic |
bug 1964986 |
| 12:45 |
pinesol |
Launchpad bug 1964986 in Evergreen "Pull list can lose pagination when a patron has more than one penalty" [Undecided,New] https://launchpad.net/bugs/1964986 |
| 12:45 |
pinesol |
News from commits: LP#1939426 Traditional Catalog Holds: Patron Info Not Populating <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=ef12f9565c8afb5b2502f6881c440782457c1155> |
| 13:04 |
|
_collum joined #evergreen |
| 13:24 |
terranm |
Bmagic Do you want a pullrequest on that patch? |
| 13:44 |
|
jihpringle joined #evergreen |
| 13:59 |
Bmagic |
terranm: I guess, but I figured I would let the community consider it |
| 14:00 |
Bmagic |
terranm: Do you get any results from the bug query? |
| 14:00 |
Bmagic |
It would be nice to know that this is a problem for more than this one system |
| 14:01 |
Bmagic |
mmorgan++ # thanks for trying it on your system |
| 14:02 |
jvwoolf |
Bmagic: We get results from that query |
| 14:05 |
Bmagic |
jvwoolf: now try the steps in the bug |
| 14:08 |
Dyrcona |
Bmagic: Adding a pullrequest tag is the best way to get the community to consider it. |
| 14:08 |
Bmagic |
oh right, ok, done |
| 14:16 |
pinesol |
News from commits: LP#1950394: isolate attribute extraction of metarecord <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=caef876670021fec21dd2e3a95951fd71c25233a> |
| 14:16 |
jvwoolf |
Bmagic: confirmed |
| 14:16 |
jvwoolf |
Is bug 1858698 related? |
| 14:16 |
pinesol |
Launchpad bug 1858698 in Evergreen "pull list: difference between displayed and printed list" [Medium,Confirmed] https://launchpad.net/bugs/1858698 |
| 14:16 |
Bmagic |
it might be! |
| 14:17 |
jvwoolf |
I will add a comment about that |
| 14:29 |
Bmagic |
A pull request for a pull request bug |
| 14:29 |
Bmagic |
haha |
| 14:29 |
csharp_ |
bugception |
| 14:52 |
terranm |
Bmagic: Sorry, was at lunch. Thanks! |
| 15:02 |
|
collum joined #evergreen |
| 16:56 |
|
mmorgan left #evergreen |
| 17:05 |
|
jihpringle joined #evergreen |
| 17:07 |
|
jvwoolf left #evergreen |
| 17:11 |
Bmagic |
I just noticed something about this pull list issue. In the examples where it's broken, I'm not getting the dropdown menu for the pickup library, even though I'm global admin, plus, the loading bar just bounces left to right, no percentage, like I see on other examples..... |
| 18:00 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 19:22 |
|
jihpringle joined #evergreen |
| 20:16 |
|
JBoyer joined #evergreen |
