| Time |
Nick |
Message |
| 00:51 |
|
jweston_ joined #evergreen |
| 00:51 |
|
JBoyer joined #evergreen |
| 00:52 |
|
bshum joined #evergreen |
| 00:52 |
|
berick joined #evergreen |
| 00:52 |
|
book` joined #evergreen |
| 00:52 |
|
degraafk joined #evergreen |
| 00:52 |
|
miker joined #evergreen |
| 00:52 |
|
abneiman joined #evergreen |
| 00:53 |
|
akilsdonk joined #evergreen |
| 00:53 |
|
Bmagic joined #evergreen |
| 00:54 |
|
troy joined #evergreen |
| 06:01 |
pinesol |
News from qatests: Failed Create Evergreen Database <http://testing.evergreen-ils.org/~live//archive/2022-02/2022-02-02_04:00:02/test.41.html> |
| 08:25 |
|
mantis1 joined #evergreen |
| 08:32 |
|
mmorgan joined #evergreen |
| 08:36 |
|
rfrasur joined #evergreen |
| 08:55 |
|
Dyrcona joined #evergreen |
| 08:55 |
|
alynn26 joined #evergreen |
| 09:00 |
Dyrcona |
miker: Well, no loops in pgtde. We do have 1,036 entries, though. I'm back where I was trying to figure it out. |
| 09:27 |
Dyrcona |
No loops in permission.grp_tree, either. |
| 10:55 |
|
Keith-isl joined #evergreen |
| 11:02 |
|
rjackson_isl_hom joined #evergreen |
| 11:05 |
Dyrcona |
Ugh. I removed [useDisplayEntries]="true" from 1 page. Did a ng build --prod; make; make install; chown, etc. |
| 11:06 |
Dyrcona |
That's it, and search is now coming up with blank entries. |
| 11:06 |
* Dyrcona |
is impatient.... :) |
| 11:06 |
Dyrcona |
I guess it took a minute or 3 after the reload. :) |
| 11:07 |
Dyrcona |
All right. It is related to our pgtde table, but there are nor loops. I'm going to check the schema to see if we're missing an index or something. |
| 11:08 |
Dyrcona |
s/nor/no/ |
| 11:14 |
Dyrcona |
Hmm.. No indexes on that table in the base schema. |
| 11:15 |
* Dyrcona |
tries adding one on org to see if that helps. |
| 11:16 |
Dyrcona |
If you're using permisison.grp_tree_display_entry and the angular catalog, I'm curious how many rows are in your permission.grp_tree_display_entry table. |
| 11:18 |
Dyrcona |
It might also be useful to know what version of PostgreSQL you're running. We're on Pg 10. |
| 11:21 |
Dyrcona |
Well, an index on org did not help..... |
| 11:21 |
Dyrcona |
Guess I'll try parent, next. |
| 11:22 |
Dyrcona |
It's definitely the permission.grp_tree_display_entry causing the freeze, though. |
| 11:26 |
jeff |
based on your description of the symptoms, i doubt it's something an index can help. it sounds like a tight loop in the browser. i'd probably debug collectGroups or sprinkle some console.log statements in/around there to see how far things get before the browser hangs. |
| 11:28 |
jeff |
but it might not be a bad idea to try the pcrud call from srfsh just to see what the data looks like / how fast it returns |
| 11:28 |
Dyrcona |
Well, here's the thing, I have done nothing to the Angular code. So it's just eg-profile-select. I suspect it's the number of entries, and maybe ejabberd blowing up along the way to the client. |
| 11:28 |
jeff |
[include whynotboth.gif] |
| 11:29 |
jeff |
(collectGroups is in Open-ILS/src/eg2/src/app/staff/share/patron/profile-select.component.ts) |
| 11:30 |
Dyrcona |
jeff: OK. |
| 11:30 |
Dyrcona |
I think we have too many profile groups anyway. |
| 11:30 |
Dyrcona |
I'm going to start with SRFSH. |
| 11:33 |
Dyrcona |
0.66 seconds... |
| 11:35 |
jeff |
same args as the Angular code uses (with org and fleshing of grp, etc)? |
| 11:35 |
Dyrcona |
Copied right from the syslog. Only thing changed was the authtoken. |
| 11:36 |
jeff |
*nod* |
| 11:37 |
Dyrcona |
I've noticed this while paging through the output: In the fleshed profile, some of our entries have __p[1] is null, some have an empty string, and others have a string there. I don't recall which field that is OTTOMH. |
| 11:38 |
Dyrcona |
Probably insignificant. |
| 11:39 |
jeff |
pgt objects? pgt array position 1 is "Description". |
| 11:39 |
Dyrcona |
jeff: Thanks! Saves me from looking it up. |
| 11:40 |
Dyrcona |
Yeah, an index isn't going to help. So I should add a breakpoint on collect groups and try again... |
| 11:40 |
Dyrcona |
Trouble is finding in the "compiled" code. |
| 11:41 |
jeff |
if you don't build in prod mode, you should have easier-to-debug code. |
| 11:41 |
Dyrcona |
Just ng build? |
| 11:41 |
jeff |
i don't know offhand, sorry. |
| 11:42 |
Dyrcona |
I'll check the docs, then. :) |
| 11:42 |
jeff |
and i also don't know offhand if you need to do anything like remove a build file. |
| 11:43 |
Dyrcona |
--prod is deprecated, but maybe not in Angular 10... |
| 11:44 |
Dyrcona |
--configuration development is what I want, I think |
| 11:45 |
Dyrcona |
Think I'll wipe out the existing code, first. |
| 11:49 |
Dyrcona |
configuration development is not set in the workspace..... |
| 11:50 |
Dyrcona |
Guess I have to use ng build --watch |
| 11:50 |
Dyrcona |
That's not really what I wanted, but OK.... |
| 11:51 |
jeff |
another option is to generate sourcemaps, but if build --configuration development didn't work I'm fairly certain I haven't done either of those two things with the Angular code in Evergreen yet, so don't take my advice too much. :-) |
| 11:52 |
Dyrcona |
I'll look at the actual configuration in a minute. |
| 11:52 |
Dyrcona |
I'm going to undo what I just did "rm -rf ../../web/eg2" :) |
| 11:54 |
|
jihpringle joined #evergreen |
| 11:55 |
Dyrcona |
It only defines two production configurations, and I'm really not that familiar with Angular, yet. |
| 11:56 |
Dyrcona |
Ah well. Lunch time. |
| 12:09 |
jeff |
how was your permission.grp_tree_display_entry table initially populated? you have so many entries, I suspect it was not populated by hand? |
| 12:10 |
|
jvwoolf joined #evergreen |
| 12:21 |
Dyrcona |
jeff: I don't know how it was populated, but I suspect it is maintained by hand. |
| 12:23 |
Dyrcona |
We have 137 permission groups and 376 org units. It could be much bigger. |
| 12:24 |
Dyrcona |
So, looks like I'm spending the afternoon with the Angular documentation so I can figure out a development config. I'll share it later. --watch is not what I want. I know, it's the greatest thing since sliced bread, but sometimes you just want to use the debugger. |
| 12:27 |
jeffdavis |
fwiw we have 301 entries in pgtde, 114 in pgt |
| 12:30 |
Dyrcona |
jeffdavis: Do any of your permission groups have empty string (not null) in the description? |
| 12:32 |
jeffdavis |
yeah, a couple dozen of them |
| 12:36 |
Dyrcona |
OK. I was just looking at the code and I think the description field is irrelevant. |
| 12:38 |
jeff |
Dyrcona: do you have output for this query? |
| 12:38 |
jeff |
SELECT DISTINCT org FROM permission.grp_tree_display_entry AS pgtde1 WHERE NOT EXISTS (SELECT 1 FROM permission.grp_tree_display_entry AS pgtde2 WHERE pgtde1.org = pgtde2.org AND pgtde2.parent IS NULL); |
| 12:38 |
berick |
Dyrcona: --watch should be all you need. the main thing for debugging is skipping the minification step, which --watch (or --dev, i thinkg) will do |
| 12:38 |
Dyrcona |
jeff: No. |
| 12:38 |
Dyrcona |
berick: I don't want --watch to hang around in my console. |
| 12:39 |
berick |
Dyrcona: then just remove --watch |
| 12:39 |
berick |
i think it does a dev build by default |
| 12:40 |
Dyrcona |
Unknown option --dev. Angular docs say a production build is default and thats what ti looks like. |
| 12:40 |
Dyrcona |
s/ti/it/ |
| 12:41 |
jeff |
i can reproduce your symptoms if i have a pgtde tree for an org where there are no rows with parent IS NULL. |
| 12:41 |
jeff |
i think it loops here: https://git.evergreen-ils.org/?p=Evergreen.git;a=blob;f=Open-ILS/src/eg2/src/app/staff/share/patron/profile-select.component.ts;h=56ff560c6047bff311d4ed1e8af80b7269a9b385;hb=refs/heads/master#l113 |
| 12:43 |
jeff |
but I'd expect you to have ourput for that query if you had an org with pgtde entries defined and none as "root" |
| 12:43 |
jeff |
(where parent is null) |
| 12:43 |
berick |
Dyrcona: 'ng build' creates dev builds for me *shrug* |
| 12:44 |
Dyrcona |
All right, well, I'll give it another go. I suspect the foreach to find the closest org is my problem. |
| 12:45 |
jeff |
good luck! let us know if you can share your pgtde entries (they're a collection of integers, so perhaps you can) |
| 12:46 |
Dyrcona |
jeff: Is that what your query was supposed to find, an entry where there is no null parent? I got 0 rows. |
| 12:47 |
|
collum joined #evergreen |
| 12:47 |
jeff |
my query was attempting to find any orgs that had pgtde entries defined, but none of the pgtde entries for that org had parent IS NULL. |
| 12:48 |
Dyrcona |
OK. |
| 12:48 |
jeff |
I intentionally created some here and was able to reproduce the symptoms you described: browser tab hangs, dev tools unresponsive, etc. |
| 12:48 |
Dyrcona |
berick: Sometimes, I'm an idiot. ng build is apparently skipping the minifying, etc. |
| 12:48 |
Dyrcona |
jeff: OK. I suspected something like that from the start, but so far have not found it. |
| 12:51 |
Dyrcona |
Now, the "fun" part... finding the file I want to debug. |
| 12:52 |
Dyrcona |
Of course, this reminds me of a time I had code that froze when run normally, but worked find in the debugger. :) |
| 12:53 |
Dyrcona |
jeff++ It's hanging on that do. |
| 12:53 |
Dyrcona |
Whee! that's seriously deep: 39578894 |
| 12:55 |
Dyrcona |
OK. I'm puzzled why this group is showing up: NCIP Staff. I don't think it's in pgtde.... |
| 12:57 |
Dyrcona |
It's in there 9 times... |
| 12:59 |
Dyrcona |
Ok. It isn't any other entry's parent in pgtde, so I'm going to delete it. |
| 13:00 |
Dyrcona |
I still have to kill the Chrome process. |
| 13:00 |
Dyrcona |
berick++ # For putting up with my ignorance. |
| 13:10 |
Dyrcona |
Still haven't figured it out, but it definitely looks like a circular reference. |
| 13:19 |
jeff |
this is a simple way to reproduce the "no top of pgtde tree" cause for the loop/hang: |
| 13:20 |
jeff |
insert into permission.grp_tree_display_entry (id, position, org, grp, parent) values (1, 1, 1, 1, 1); |
| 13:20 |
jeff |
(starting with an empty pgtde table) |
| 13:20 |
Dyrcona |
OK.... I'm starting to suspect that I have an entry somewhere that doesn't hit the top of the tree, either in pgt or pgtde. |
| 13:21 |
Dyrcona |
I don't think I need to reproduce it. I just need to figure out which entry or entries has this problem. |
| 13:21 |
Dyrcona |
miker's query didn't turn anything up. |
| 13:24 |
jeff |
"\copy permission.grp_tree_display_entry to pstdout"? :-) |
| 13:24 |
Dyrcona |
tmp = groups.filter(g => g.id() === pid)[0]; This never gets null or false. |
| 13:25 |
Dyrcona |
also the earlier break never gets hit, either. |
| 13:26 |
Dyrcona |
Maybe. I dumped pgt and pgtde to csvs the other day. |
| 13:28 |
Dyrcona |
Ok. That code I pasted should always produce a value. I've got a set of pgtde entries that never hit the top of the tree. |
| 13:30 |
Dyrcona |
If I combine miker's query and compare the parents array to my csv, I should be able to figure it out. I'm looking for an array that doesn't contain one of the nodes with a null parent. |
| 13:35 |
jeff |
note that miker's query as written above starts with a pgtde entry that has parent IS NULL |
| 13:38 |
jeff |
this has a root of tree but also a loop: |
| 13:39 |
jeff |
INSERT INTO permission.grp_tree_display_entry (id, parent, all_parents, depth) VALUES (1, 1, 1, 1, NULL), (2, 1, 1, 2, 3), (3, 1, 1, 3, 2); |
| 13:42 |
Dyrcona |
I can also just remove about 26 charactes from the code and this problem "goes away." :) |
| 13:42 |
Dyrcona |
Yeah, I should find the loop. |
| 13:42 |
jeff |
see if this finds anything? |
| 13:42 |
jeff |
with recursive tree as (select id,parent,array[id] as all_parents, 0::int as depth from permission.grp_tree_display_entry union all select c.id,c.parent,p.all_parents||c.id, p.depth+1 from permission.grp_tree_display_entry c join tree p on (c.parent = p.id) where depth < 1000) select * from tree WHERE depth > 999; |
| 13:43 |
jeff |
(modified miker's query, tested it on my little test case) |
| 13:44 |
Dyrcona |
jeff: Your query returns 0 rows on my data. I wonder if the problem is pgt? |
| 13:44 |
jeff |
wacky. |
| 13:45 |
Dyrcona |
We have multiple nodes with null parent in pgtde. |
| 13:46 |
jeff |
for any org unit with pgtde entries, you should have at least one pgtde entry that has a null parent. based on you running one of my queries above, you seem to be okay there. |
| 13:47 |
Dyrcona |
Here is pgtde.csv: https://pastebin.com/kg4tWqwZ |
| 13:47 |
jeff |
and you can have multiple pgtde entries with null parent, ideally one and only one null parent pgtde per org (with pgtde entries) |
| 13:48 |
Dyrcona |
Yeah, I can't find anything wrong with the data or the code. All I know is, if I remove useDisplayEntries from the Anuglar patron search, it doesn't hang. With that option, it hangs on our data. |
| 13:51 |
Dyrcona |
Unrelated: You ever go on pastebin and just look at some of the recent pastes? You can find some strange stuff. |
| 13:53 |
* miker |
reads up |
| 13:54 |
miker |
ah, good thinking jeff! my query only considered "sane-top" trees with loops further down, not the "perfect circle" tree |
| 13:55 |
Dyrcona |
I just checked the orgs that appear in pgtde, and they all have group 1 with a null parent. |
| 13:56 |
Dyrcona |
Should the pgtde parent entries match the pgt entries' parents? |
| 14:01 |
jeff |
no. |
| 14:01 |
jeff |
pgtde.parent refers to pgtde.id |
| 14:01 |
jeff |
pgt.parent refers to pgt.id |
| 14:13 |
|
Dyrcona joined #evergreen |
| 14:19 |
|
jihpringle joined #evergreen |
| 14:22 |
Dyrcona |
jeff: I was just wondering though if Group B is child of GroupA in pgt, but they're backwards in pgtde. It doesn' look like it should matter, but... |
| 14:25 |
|
Keith_isl joined #evergreen |
| 14:27 |
|
mantis2 joined #evergreen |
| 14:55 |
|
jihpringle joined #evergreen |
| 15:27 |
Dyrcona |
So, that code is looping over the groups. It looks like the problem is with my grp_tree and not grp_tree_display_entry. |
| 15:35 |
|
Keith_isl joined #evergreen |
| 15:40 |
Dyrcona |
I'm starting to wonder if the order that the data comes out may matter. Could this be a place where behavior has changed with Pg 10? |
| 15:40 |
|
jvwoolf1 joined #evergreen |
| 15:42 |
|
jvwoolf1 left #evergreen |
| 16:13 |
|
JBoyer joined #evergreen |
| 16:13 |
|
jweston_ joined #evergreen |
| 16:13 |
|
rfrasur joined #evergreen |
| 16:13 |
|
collum joined #evergreen |
| 16:13 |
|
jeff joined #evergreen |
| 16:13 |
|
rhamby joined #evergreen |
| 16:13 |
|
dbs joined #evergreen |
| 16:13 |
|
phasefx joined #evergreen |
| 16:13 |
|
jihpringle joined #evergreen |
| 16:13 |
|
dluch_ joined #evergreen |
| 16:13 |
|
jeffdavis joined #evergreen |
| 16:13 |
|
ejk joined #evergreen |
| 16:13 |
|
Dyrcona joined #evergreen |
| 16:13 |
|
Christineb joined #evergreen |
| 16:13 |
Dyrcona |
On the wrong side of the net split. Just you and me, Christineb. |
| 16:13 |
|
mmorgan joined #evergreen |
| 16:13 |
|
akilsdonk joined #evergreen |
| 16:13 |
|
Bmagic joined #evergreen |
| 16:14 |
|
pinesol joined #evergreen |
| 16:14 |
|
bshum joined #evergreen |
| 16:14 |
|
berick joined #evergreen |
| 16:14 |
|
book` joined #evergreen |
| 16:14 |
|
degraafk joined #evergreen |
| 16:14 |
|
miker joined #evergreen |
| 16:14 |
|
abneiman joined #evergreen |
| 16:14 |
|
csharp_ joined #evergreen |
| 16:14 |
|
eby joined #evergreen |
| 16:15 |
|
troy joined #evergreen |
| 16:15 |
|
gmcharlt joined #evergreen |
| 16:15 |
|
jonadab joined #evergreen |
| 16:17 |
Dyrcona |
Bleh. I hate stuff like this. "It's broken." "No, it isn't." "Yes, it is." "OK. It is, but it shouldn't be." |
| 16:22 |
jeff |
Dyrcona: did you see my last, "(this is in addition to..."? |
| 16:23 |
jeff |
hrm. looks like it's missing from logs, at least. |
| 16:23 |
jeff |
16:05 < jeff> if using pgtde, tmp = groups.filter(g => g.id() === pid)[0]; is referencing the wrong thing. |
| 16:24 |
jeff |
16:05 < jeff> if you change it to tmp = groups.filter(g => g._display.id() === pid)[0]; it doesn't loop, but now it's likely going to break when using pgt-not-pgtde. |
| 16:24 |
jeff |
16:05 < jeff> do you have a launchpad bug already? i can open one. |
| 16:24 |
jeff |
16:06 < jeff> Dyrcona: do you have a launchpad bug already? i can open one. |
| 16:24 |
jeff |
16:06 < jeff> (this is in addition to the looping that can happen with "there is no root entry" and "there is a loop") |
| 16:24 |
Dyrcona |
jeff: I didn't get that, net split. |
| 16:24 |
Dyrcona |
But, it's not using pgtde at that point. It's using the fleshed groups. |
| 16:25 |
Dyrcona |
In the debugger, the type shows as pgt. |
| 16:26 |
Dyrcona |
How come it doesn't seem to break for anyone else, then? |
| 16:26 |
Dyrcona |
I'll edit the code in the debugger and see what happens. |
| 16:28 |
Dyrcona |
ugh.... |
| 16:28 |
Dyrcona |
This has been painful. |
| 16:28 |
Dyrcona |
Dealing with a stuck browser when breakpoints are ignored. Supposedly there's a chrome task manager now, too. |
| 16:29 |
Dyrcona |
They might as well just slap it on a laptop and call it an operating system.... Oh, wait.... |
| 16:31 |
Dyrcona |
Yeah, that change doesn't seem to fix it for me. It still loops on group 344. |
| 16:32 |
jeff |
yes, i've been using the chrome task manager to kill just the tab. |
| 16:32 |
jeff |
also, I put in a break in the loop when depth > 10 so it would stop hanging while I was debugging. |
| 16:32 |
jeff |
hrm. interesting! |
| 16:33 |
jeff |
your line reads this, now? |
| 16:33 |
jeff |
tmp = groups.filter(g => g._display.id() === pid)[0]; |
| 16:34 |
Dyrcona |
tmp = groups.filter(g => g._display.id() === pid)[0]; |
| 16:34 |
Dyrcona |
I saved it, too. |
| 16:34 |
Dyrcona |
I'm also using rel_3_7 if that matters. |
| 16:34 |
jeff |
if you are editing the source in the Chrome debugger, are you editing it while paused, then saving, then continuing/resuming? |
| 16:34 |
jeff |
hrm. |
| 16:35 |
Dyrcona |
I edited it before hitting patron search this last time. |
| 16:35 |
Dyrcona |
I'll remove the breakpoint on that line and try again. |
| 16:36 |
jeff |
I'm not editing in the debugger, just rebuilding each time. |
| 16:37 |
Dyrcona |
Ok. I'll try that this time. |
| 16:38 |
Dyrcona |
Hah. I read "Wall ops from A_Dragon" as "wallops from a dragon" which I think is an intentional pun. |
| 16:40 |
Dyrcona |
jeff: I think something like that ternary operator where pid is set would work to fix it. |
| 16:41 |
Dyrcona |
=> is a lambda, isn't it? |
| 16:45 |
Dyrcona |
jeff++ # That fixes it for me. I'll open a Lp bug tomorrow and come up with a decent patch. |
| 16:52 |
jeff |
the reason your data was looping was because eventually it's looking at display entries and trying to find parents, but it's taking the pgtde.parent value and looking at pgt.id. |
| 16:54 |
jeff |
eventually it gets to a point where it finds a display entry whose display entry parent points at a pgt row that exists in pgtde and whose pgtde.parent happens to be equal to its pgt.id |
| 16:54 |
jeff |
and since it's looking at the wrong id when trying to walk up the tree, it loops. |
| 16:56 |
Dyrcona |
So, you're saying that the problem is pgtde.grp == pgtde.parent? |
| 16:56 |
jeff |
Dyrcona: if you run the original unmodified code but delete permission.grp_tree_display_entry with id 223 (which has grp = 120 and parent = 120), you will likely not loop/freeze. |
| 16:57 |
jeff |
yes. pgtde.grp = pgtde.parent SHOULD NOT be a problem, but since the code is looking at the wrong value when walking up the tree, it IS currently a problem. |
| 16:57 |
jeff |
there was a loop, it was just a loop across two tables :-) |
| 16:57 |
jeff |
(that are supposed to be two different trees, but due to the bug were... getting twisted up) |
| 16:59 |
jeff |
i loaded your pgtde data and inserted dummy org units and permission groups and could immediately reproduce the issue, so 1) i didn't need to grab your pgt or aou data and 2) i was pretty sure it wasn't something with your pgt or aou tree structure |
| 17:01 |
Dyrcona |
OK. I see it. I'll open a Lp bug tomorrow. I think we can patch it with a ternary operator that will work on _display if it is there. |
| 17:01 |
Dyrcona |
jeff++ |
| 17:01 |
jeff |
yeah, I opted for a more traditional if/else block, but that probably also works! |
| 17:01 |
Dyrcona |
Well, if you have a patch, feel free to submit it. |
| 17:01 |
jeff |
I'm going to see about fixing some of the other issues. rather have "incorrectly indented" entries than "browser hangs forever" :-) |
| 17:02 |
Dyrcona |
I didn't see it because the values in the debugger weren't showing that entry, but others. |
| 17:02 |
Dyrcona |
Anyway, look at the time! |
| 17:13 |
mmorgan |
jeff++ |
| 17:17 |
|
mmorgan left #evergreen |
| 18:02 |
pinesol |
News from qatests: Failed Installing Angular web client <http://testing.evergreen-ils.org/~live//archive/2022-02/2022-02-02_16:00:02/test.29.html> |
