| 10:22 |
|
sleary joined #evergreen |
| 10:23 |
Dyrcona |
xmllint and namespaces..... :( |
| 10:25 |
Dyrcona |
Remove the namespace and I still get XPath set is empty.... |
| 10:27 |
Dyrcona |
Can I pick records to test, or what? <datafield tag="024" ind1=" " ind2=" "><subfield code="">660355362927</subfield></datafield> |
| 10:27 |
Dyrcona |
No subfield code..... |
| 10:27 |
Dyrcona |
How's that even possible? |
| 10:30 |
Dyrcona |
And, another one with a blank subfield code. |
| 12:51 |
Dyrcona |
OK. I got back to messing with my UPCs and I've verified that the XPath should work. Guess I'll have to wait for the full ingest to finish. One of the options that I used yesterday must have prevented the identifiers from being handled. |
| 12:55 |
Dyrcona |
Oof! |
| 12:55 |
Dyrcona |
I see I still have a problem and the XPath that I updated won't work.... |
| 12:58 |
Dyrcona |
I had to modify it slightly for the test script, and I assumed it was equivalent to what was in the database. It was not. |
| 12:59 |
Dyrcona |
OK! Now, it works with the same syntax. |
| 13:00 |
Dyrcona |
Too many different domain specific languages to try and keep straight.... "Says you..." "Says who?" "Says you, the Lisp guy." |
| 13:00 |
Dyrcona |
There's a joke in there. I promise. :) |
| 13:22 |
Dyrcona |
And, here come the "closing early" emails on the directors' list... :) |
| 13:54 |
jeff |
just perpetually overcast and gray skies here: https://youtu.be/FDvImJ_6HYw |
| 15:12 |
Dyrcona |
Looks like I finally fixed the XPath. I've got 106 entries in identifier_field_entry and they all look like UPCs. The ingest hasn't finished, yet. |
| 15:44 |
jeffdavis |
I've got a 3.9.1 test server where I've generated search suggestions (ran the sideloader steps and populated search.symspell_dictionary with >6M rows) and have the org settings/internal flags set to default values, the but I'm not seeing any suggestions on my test searches. Are there other steps I need to take to make this work? |
| 15:58 |
Dyrcona |
jeffdavis: Are you doing single word searches? |
| 16:00 |
jeffdavis |
good question - I am, I'll double check with other testers |
| 16:05 |
jihpringle |
jeffdavis: I just tried a single word and then a multiple word search on our testing server and I don't see anything new whether I get results or not |
| 16:49 |
Dyrcona |
Sorry, got distracted by some work. :) |
| 16:50 |
Dyrcona |
It has been a while since I looked at it. I'll have to do some code diving tomorrow. |
| 16:50 |
Dyrcona |
Cheers! |
| 14:31 |
jeff |
can you explain what you mean by "do not look like UPCs"? also, depending on a number of things, you might have a lot of outliers in that LIMIT 100... |
| 14:31 |
Dyrcona |
Yeah, when I do field = 18 they look like ISBNs, and 18 is ISBN. |
| 14:32 |
Dyrcona |
jeff: No problem: 192326039 | 4144895 | 20 | 1. Selected Economic Indicators, 2014-212. Financial Soundness Indicators, 2010-16; ANNEXES; I. Risk Assessment Matrix; II. Report on the Observance of Standards and Codes (ROSCs)-Summary Assessments of BCP; III. Report on the Observance of Standards and Codes (ROSCs)-Summary Assessments of ICP; IV. Report on the Observance of Standards and Codes (ROSCs)-Summary Assessments of C |
| 14:32 |
Dyrcona |
PFMI; V. Previous FSAP Recommendations; VI. Banking Stress Testing Matrix; VII. Corporate Stress Testing Matrix. 1. The FSCFIGURES; 1. The Macro Context; 2. A Bank Dominated Financial Sector; 3. Banking Sector Profile; 4. Bank Business Model Convergence; 5. Bank Measures of Systemic Risk and Spillover Networks; 6. Banks' FX and Liquidity Risks; 7. Banks' Returns, Asset Quality, and Solvency; 8. Corporate Sector Risks and Vul |
| 14:32 |
Dyrcona |
ies; 9. Summary Results of Solvency Stress Tests of Major Banks; 10. Results of the Liquidity Stress Tests of Major Banks; 11. Funding Liquidity-Solvency Feedback in Solvency Stress Tests; 12. Corporate Sector Stress Test Analysis; TABLES. |
| 14:32 |
Dyrcona |
That's definitely NOT a UPC. |
| 14:32 |
jeff |
ah, yes. |
| 14:32 |
Dyrcona |
That's the first result row. I must have a conflicting definition somewhere. |
| 14:36 |
Dyrcona |
So, we've got 505 and 246 possibly showing up. |
| 14:37 |
Dyrcona |
I used to know this part of Evergreen better than I do now. |
| 14:38 |
Dyrcona |
jeff++ mmorgan++ |
| 14:39 |
Dyrcona |
This is on a development/test system. I guess I'll double check production, too. Maybe it was something in this upgrade script.... |
| 14:39 |
jeff |
nothing stock references 505, but in mods that's tableOfContents, which is matched by the xpath for the keyword field "toc". |
| 14:40 |
jeff |
also, that specific record is deleted in your live system, which may or may not mean that its being skipped on reingests/etc. |
| 14:40 |
jeff |
s/its/it's/ |
| 10:02 |
|
Dyrcona joined #evergreen |
| 12:39 |
|
jihpringle joined #evergreen |
| 13:56 |
|
dmoore joined #evergreen |
| 14:01 |
mantis1 |
HI all. Pushed a working branch to the EG repo but can't find it/can't check the branch out in my test box |
| 14:01 |
mantis1 |
command was git push working lp1853630_carousel_shelving_location_desc:user/gmonti/lp1853630_carousel_shelving_location_desc |
| 14:02 |
mmorgan |
mantis1: it |
| 14:02 |
mmorgan |
's in the working repo: https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/gmonti/lp1853630_carousel_shelving_location_desc |
| 14:02 |
mantis1 |
ah thank you |
| 14:02 |
mmorgan |
did you do a git fetch working? |
| 14:02 |
mantis1 |
I did sorry I was looking in the wrong repo |
| 14:02 |
mantis1 |
Thank you! |
| 14:03 |
mantis1 |
Do we add a sign off tag to the LP ticket? |
| 14:04 |
mantis1 |
or is that just after testing? |
| 14:05 |
Dyrcona |
mantis1: If you signoff the branch, all you have to do on the ticket is say that you pushed a signoff branch and where the branch is. |
| 14:05 |
Dyrcona |
It's probably best to wait until after you've tested it. I don't add my signoff if I can't get it to work. |
| 14:05 |
mmorgan |
mantis1: It needs a pullrequest tag, the signoff tag gets added by the tester |
| 14:05 |
Dyrcona |
Oh, yeah. You can add the signoff tag if you tested it. |
| 14:06 |
Dyrcona |
I may have misunderstood the situation. |
| 14:06 |
Dyrcona |
The original author doesn't add the signoff tag for their signoff. |
| 14:06 |
mantis1 |
mmorgan++ Dyrcona++ |
| 15:08 |
Dyrcona |
jihpringle++ jeffdavis++ |
| 15:08 |
terranm |
jihpringle++ jeffdavis++ |
| 15:08 |
Dyrcona |
Some of those will slow things down. |
| 15:08 |
jeffdavis |
Testers could also use it during bug squashing weeks, people doing acceptance testing on paid development project, etc. |
| 15:09 |
sandbergja |
3 and 7 also seem difficult. We don't really have comprehensive information about how all our functionality/settings are supposed to work. |
| 15:09 |
sandbergja |
be it 100% test coverage or a super-detailed manual |
| 15:10 |
JBoyer |
I do like the idea of basing an interface comparison checklist on it. You don't necessarily need to know how to use all of an interface if you can tell that you can get to X on dojo but not Angular. (so long as it's not an intentional workflow change) |
| 15:10 |
sandbergja |
so I'd be interested in some conversation about how developers can get all that info before embarking on an angularization |
| 15:13 |
Dyrcona |
I wonder if the Angular version needs to have feature parity with the previous interfaces(s). What if we came up with better ways of doing things? |
| 15:14 |
mmorgan |
re: library settings - it seems like calls to get library settings in existing code shouldn't be too hard to identify. |
| 15:14 |
JBoyer |
That's what I was getting at yeah. If you can get the same outcome with a different UI that's fine, but being able to do something in the old and busted but not the new hotness isn't great. |
| 15:15 |
tlittle |
#info tlittle = Tiffany Little, PINES |
| 15:17 |
jeffdavis |
IMO the state of our tests/docs means that we'll never catch all of these problems. I think the idea with a checklist was to make sure we're consciously thinking about these types of things at some point during the contribution process. |
| 15:17 |
berick |
yeah, what jeffdavis said |
| 15:17 |
Dyrcona |
This is going to be unpopular, but maybe we need stricter standards than "works for me" and X number of signoffs. |
| 15:18 |
sandbergja |
Dyrcona: what did you have in mind? |
| 15:19 |
berick |
so more emphasis is good |
| 15:19 |
berick |
and having a record of usual gotchas helps focus |
| 15:20 |
abneiman |
this is also where non-technical end users can be helpful - in terms of interface & workflow evalution |
| 15:20 |
Dyrcona |
sandbergja: I'm not really looking for more process, but I do think we need better automated tests, etc. I don't have much specific in mind at the moment. |
| 15:20 |
mmorgan |
abneiman++ |
| 15:21 |
Dyrcona |
Also, what abneiman said. We need more end users involved. |
| 15:21 |
tlittle |
abneiman++ |
| 15:22 |
mmorgan |
It's difficult for end users to get more involved, difficult for them to get their hands on the new development to test on. |
| 15:22 |
JBoyer |
Yeah, testers rarely need to be conversant in the technical bits, and sometimes the tech types don't know so much about how the end users do their thing. |
| 15:23 |
mmorgan |
Bugsquashing week is huge, but test systems are not real world. |
| 15:23 |
gmcharlt |
yeah - I think an additional factor is committer time, and more automation can help around the edges |
| 15:23 |
tlittle |
Numbers 6-7 are concrete things that fall squarely on the dev's shoulders, though, imo |
| 15:24 |
mmorgan |
tlittle++ |
| 15:25 |
JBoyer |
But to mmorgan's point, it's difficult to test on realistic data if you don't have the local staff to build and rebuild test systems. :-/ |
| 15:25 |
Dyrcona |
Well, hire someone. What we need is more resources. It's that simple. |
| 15:26 |
terranm |
Yeah, we find a lot of issues when we do our intensive pre-upgrade testing with a copy of real data. And then more issues when we go live. |
| 15:27 |
mmorgan |
Dyrcona: Agreed more resources, but imo it doesn't seem that simple to hire someone and bring them up the Evergreen learning curve. |
| 15:29 |
berick |
it may be helpful to consider that this phase of EG dev won't last forever. |
| 15:29 |
berick |
there's only so many non-Angular UI's |
| 15:31 |
dmoore |
oh duh, thanks |
| 15:31 |
sleary |
Even just focusing on the Angular UIs is a steep learning curve. I have had trouble with the lack of inline documentation on what various components do. (This topic is on the next new dev agenda, btw.) |
| 15:31 |
csharp_ |
dmoore: feel free to ask if there are other acronyms/jargon you don't understand |
| 15:32 |
sandbergja |
Dyrcona: for me, I think that goes back to better automated tests and docs -- hopefully when/if we migrate away from Angular, we'd have better safeguards against regressions |
| 15:32 |
mmorgan |
All part of the learning curve :) |
| 15:32 |
csharp_ |
part of the answer is new devs - both in general and the new devs group |
| 15:32 |
sandbergja |
also, here's hoping Angular stays healthy for quite some time yet. :-) |
| 15:34 |
csharp_ |
jihpringle++ jeffdavis++ |
| 15:34 |
abneiman |
yes, jihpringle++ jeffdavis++ I have already bookmarked this |
| 15:34 |
terranm |
Same here |
| 15:34 |
sleary |
Very useful! And I would like to do more with automated testing as well. |
| 15:34 |
mmorgan |
Agreed QA gotchas will be useful, there will always be change, we just need to manage it well |
| 15:35 |
mmorgan |
*just* |
| 15:35 |
jvwoolf |
jihpringle++ jeffdavis++ |
| 15:35 |
JBoyer |
So, I do think it should go to the dev lists (perhaps quarterly or so, even) so that we can move on to the second action item from the last meeting. :) |
| 15:35 |
Dyrcona |
This list is a good outline of where we need to pay attention, and tests/standard can be built around it. |
| 15:35 |
JBoyer |
Which is |
| 15:35 |
JBoyer |
#info Bmagic to email the development list about a way to share common Evergreen tools with the community. |
| 15:36 |
JBoyer |
Though I don't think Bmagic is available at the moment and I don't recall seeing this on the lists. We'll kick that can down the road. |
| 15:39 |
JBoyer |
Note though, if anyone else has an interest in the expanded sample data set and opinions on integration and etc. feel free to poke around, |
| 15:40 |
JBoyer |
Ugh, look at me, forgetting to add something to the agenda myself, heh. |
| 15:40 |
JBoyer |
#topic Evergreen Release Updates |
| 15:40 |
JBoyer |
#info I've built a 3.8.2 release to end the 3.8 line now that 3.10 is out, if you can help test it's at https://evergreen-ils.org/downloads/Evergreen-ILS-3.8.2.tar.gz (Note, I'm going to have to make a docs change before it's final-final, but no code changes.) |
| 15:41 |
gmcharlt |
JBoyer++ |
| 15:41 |
shulabear |
Jboyer++ |
| 15:41 |
sandbergja |
JBoyer++ |
| 15:41 |
mmorgan |
JBoyer++ |
| 15:41 |
Dyrcona |
JBoyer++ |
| 15:41 |
JBoyer |
It is 1. wicked late, and 2. not too difficult to test. IF you have access to a 3.8.1 db especially and can help test it (I know the version upgrade is fine for concerto, but you know how that goes) |
| 15:41 |
terranm |
JBoyer++ |
| 15:41 |
tlittle |
JBoyer++ |
| 15:42 |
rfrasur |
JBoyer++ |
| 15:48 |
mmorgan |
jeffdavis: Yes, mostly without branches. A few have pullrequests |
| 15:49 |
jvwoolf |
mmorgan: I've push patches to all of the ones that were showstoppers for us so far |
| 15:49 |
terranm |
We're going ahead with upgrading because they're not show stoppers for us, but we know there will be some grumblings |
| 15:49 |
jvwoolf |
I'm likely done with my concentrated effort unless something comes up when we widen our testing to end users, or after we upgrade |
| 15:50 |
mmorgan |
Template issues, silent failure issues are big ones for us. |
| 15:51 |
mmorgan |
We're still sorting through and prioritizing |
| 15:51 |
|
smorrison joined #evergreen |
| 10:10 |
Dyrcona |
This JabberClient instance is no longer connected to the server |
| 10:13 |
Dyrcona |
ejabberd.log is, of course, useless. |
| 10:26 |
miker |
Dyrcona: I have ideas for making facets always faster (or at least never slower) since they can be turned into unique int IDs, but tuits... |
| 10:29 |
Dyrcona |
miker: Sounds cool. I also think that should be a separate bug from the ones I'm testing. It's just something that I've noticed. |
| 10:30 |
Dyrcona |
It's not always slower, just sometimes. |
| 10:30 |
* Dyrcona |
goes fishing through logs for a really crazy search to test. |
| 10:45 |
Dyrcona |
So, that jabber thing that I reported earlier: Has anyone ever seen services crash and have 8 listeners all using 100% cpu? This appears to have happened right after I restarted services this morning. |
| 10:47 |
Dyrcona |
I suspect something in the configuration must be busted. |
| 10:57 |
Dyrcona |
Huh. Just restarting ejabberd seems to have fixed it for now. |
| 12:41 |
jvwoolf |
(We don't have symspell in our current version, still on 3.6.5) |
| 12:49 |
pinesol |
News from commits: Docs: updating Global Flags docs <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=8326611f8dc232dda993dca412e429713189999a> |
| 13:49 |
csharp_ |
jvwoolf: it's definitely quicker without all the triggers if you can spare the downtime |
| 13:54 |
jvwoolf |
csharp_++ |
| 13:55 |
jvwoolf |
We have some time so I might test still test with triggers and without |
| 13:56 |
jvwoolf |
My guess is that the libraries may prefer to be offline than to have their Sunday staff let loose on a new version that might be slow because of the reingest, but I'll ask that question to our user group |
| 14:05 |
Dyrcona |
I think there are some patches in working branches that can make the ingest more palatable, but it looks like I have it on my other laptop, unless the one I was looking at went in. |
| 14:07 |
jvwoolf |
Dyrcona: The fix for bug 1931737 is in 3.9.1 |
| 14:07 |
pinesol |
Launchpad bug 1931737 in Evergreen 3.8 "Did you mean breaks parallel reingest and causes deadlocks when loading/overlaying bib records in the client" [High,Fix committed] https://launchpad.net/bugs/1931737 |
| 11:27 |
berick |
BTW, not sure if the backend will cache anything for a search that times out |
| 11:28 |
Dyrcona |
Well, it seems to work in the OPAC most of the time. |
| 11:29 |
Dyrcona |
I'll have to compare the two in more depth later. |
| 11:29 |
* Dyrcona |
is testing some search fixes on production data. |
| 11:31 |
Dyrcona |
It might be this particular search is way too broad given our data, but this has been consistent with other searches, IIRC. |
| 11:32 |
Dyrcona |
This search isn't returning results in the OPAC, either, and it gives up almost immediately. |
| 11:33 |
Dyrcona |
The query doesn't run nearly as long in the database now, either. |
| 11:56 |
|
jvwoolf joined #evergreen |
| 12:06 |
miker |
not here, but staff search never uses the cache. else newly added records wouldn't show up |
| 12:16 |
miker |
in the olden days, the #staff modifier would force skipping the cache. I think that still works? but docache probably overrides that heuristic |
| 12:20 |
Dyrcona |
Well, I've done some more fiddling around with that bug and comparing produciton with my test system with search patches. I get results in production regardless of filters, etc. On the test system, I either have to change to a title search or limit by library to get results. |
| 12:20 |
Dyrcona |
I suppose it is just my test database being slower than production. |
| 12:26 |
|
dguarrac joined #evergreen |
| 12:34 |
|
collum joined #evergreen |
| 12:47 |
|
collum joined #evergreen |
| 10:38 |
Dyrcona |
Not bad though, 3.7 seconds for a seq scan on 3.7 million records. |
| 10:39 |
Bmagic |
better than a human |
| 10:43 |
Dyrcona |
Adding an index on bre.vis_attr_vector doesn't make a difference in the explain output for the simple query. |
| 10:44 |
Dyrcona |
Also, there's an error in the SQL I pasted before. I'm actually running this to test: select id from biblio.record_entry where vis_attr_vector IS NULL OR NOT int4range(0,268435455,'[]') @> ANY(vis_attr_vector); |
| 10:44 |
Dyrcona |
I'm going to add an index on acvac.vis_attr_vector and try the big query again. |
| 10:47 |
Dyrcona |
That doesn't seem to make a difference either, so I'll try the procedural index that miker suggested yesterday next. |
| 10:54 |
Dyrcona |
"create index bre_not_deleted_idx ON biblio.record_entry (id) where deleted = 'f';" doesn't seem to help, either. |
| 11:14 |
miker |
select * from config.internal_flag where name like '%delete%'; if you're in psql |
| 11:16 |
Dyrcona |
I'm pretty sure that's on, but I'll double check. |
| 11:17 |
Dyrcona |
Nope. enabled = 'f'. |
| 11:17 |
miker |
kk |
| 11:20 |
miker |
Dyrcona: a big change in pg 12 is the default for non-materialized CTEs. we can't just do it across the board yet, but do you have the full original query handy to test by hand? if so, in front of each CTE (WITH-clause), after the AS keyword, what happens if you add MATERIALIZED between AS and the open-( ? |
| 11:21 |
Dyrcona |
I'll give that a shot. I have it open in my editor. |
| 11:23 |
miker |
if that makes things happy -- and I can imagine it will because estimates of the number rows coming out of a tsearch CTE are ... usually way off -- then we /can/ add a PG version check and add it when it's available. there's precedent for that in the record attribute testing infrastructure. |
| 11:27 |
Dyrcona |
It's worse, but I may have missed a couple: Execution Time: 115367.485 ms |
| 11:31 |
Dyrcona |
OK! I missed c_attr and b_attr before. after adding materialized to those two, it was much better: Execution Time: 3123.997 ms |
| 11:32 |
Dyrcona |
My guess is c_attr as materialized does the magic. |
| 11:34 |
Dyrcona |
"That's a bingo!" |
| 11:37 |
miker |
limit and offset are usually optimization fences ... Dyrcona, would you mind trying this? remove the MATERIALIZED on c_attr and add "OFFSET 0" just before the closing ) on that CTE |
| 11:38 |
Dyrcona |
OK. I was about to say that we'd have to bump the minimum Pg version to 12 if we use materialized CTEs. |
| 11:38 |
miker |
if that has the same effect, we should be able to (for now, until PG learns how to ignore useless limit/offset) do that. |
| 11:39 |
miker |
as for versioning, we already test for the version in one other place and change query structure based on that. we could do it here if MATERIALIZED is the answer |
| 11:40 |
miker |
we could also mark the function volatile to force materialization |
| 11:40 |
Dyrcona |
Execution Time: 2675.806 ms |
| 11:40 |
miker |
oh hey, look at that |
| 11:40 |
Dyrcona |
So, the offset works, too. |
| 11:44 |
Dyrcona |
Should I try that, too, or are we happy with "offset 0"? |
| 11:44 |
miker |
it is, but only in OPAC searches. staff search does use search.calculate_visibility_attribute_test thought |
| 11:45 |
miker |
though, even |
| 11:45 |
Dyrcona |
FYI, I'm testing on Pg 13, but it probably doesn't matter. |
| 11:47 |
miker |
offset 0 is the more self-documenting variant that works for all pg version, for now. though, again, we can version-test and use materialized in 12+ |
| 11:49 |
miker |
testing offset 0 on an older version, just to be sure |
| 11:49 |
Dyrcona |
I was talking about marking search.calculate_visibility_attribute_test as volatile. |
| 11:50 |
miker |
right |
| 11:50 |
Dyrcona |
I'm going to try it just because I can. |
| 11:50 |
miker |
+1, try away! |
| 11:52 |
Dyrcona |
Volatile without offset 0 works, too, but I'll defer to your opinion on offset 0. |
| 11:53 |
Dyrcona |
Execution time is about the same, 4ms difference which is probably not related to the change. |
| 12:17 |
miker |
I got curious and checked the difference between ts_rank and ts_rank_cd ... it's bigger than I thought :( |
| 12:18 |
miker |
in my test query, 3ms vs 90ms, respectively |
| 12:22 |
|
jihpringle joined #evergreen |
| 12:28 |
miker |
Dyrcona: do you want to branchify that, or shall I? |
| 12:28 |
Dyrcona |
miker: ts_rank makes a roughly 67ms difference in execution time. |
| 12:28 |
Dyrcona |
in my test environment. |
| 12:29 |
Dyrcona |
If you want to branch it, that's fine with me. I was going to open a Lp bug, but I should actually look into yet another OOM on the db server. |
| 12:29 |
Dyrcona |
It happened just before noon. |
| 12:29 |
miker |
oh, well, that's not enough to warrant further investigation this moment. my test case may be unusual |
| 12:29 |
miker |
I'll put together the patch and await your LP, then? |
| 12:30 |
Dyrcona |
OK with me. |
| 12:36 |
Dyrcona |
Is it possible to POST an OPAC search? |
| 12:42 |
miker |
it is |
| 12:43 |
miker |
(and, note, i was incorrect about staff search not using the patron vis test function -- it does, it just ignores that the the WHERE clause) |
| 12:43 |
miker |
Dyrcona: https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/optimization-fence-for-pg-12-CTEs-in-search for your eyeballs |
| 12:45 |
Dyrcona |
miker++ |
| 12:46 |
Dyrcona |
Re POST: I guess we'll need to work out a mitigation for POSTS with "bad" queries in them. |
| 13:33 |
|
kworstell-isl joined #evergreen |
| 15:21 |
jihpringle |
Bmagic: depends on the version, there's an admin interface starting in 3.8 or 3.9 I think (we definitely have it in 3.9) |
| 15:21 |
Bmagic |
jihpringle++ |
| 15:22 |
mmorgan |
It's new in 3.9 |
| 15:22 |
Stompro |
I testing on 3.9, so I get to try that out. |
| 15:25 |
jeffdavis |
In addition to the new staff portal config options, we add an iframe to the page so we can display news updates from another source: http://git.sitka.bclibraries.ca/gitweb/?p=sitka/evergreen.git;a=commitdiff;h=45bfec6b5b1a7a8d0dfd898aa68926d17bb8f1ca |
| 15:25 |
jeffdavis |
I should probably turn that into a feature request. |
| 15:28 |
Stompro |
jeffdavis, thanks, that is interesting. |
| 15:31 |
|
kworstell-isl joined #evergreen |
| 15:34 |
Stompro |
Shoot, "it won’t try to merge branch, system, and consortial-level entries" I cannot have System portal with the odd entry for a specific branch. But I get that would be more complex to figure out. |
| 15:53 |
jihpringle |
ya, if you want different portals for different org units with slight differences you have to re-create the portal for each org unit you want (but there is a clone button) |
| 15:58 |
miker |
Dyrcona: I'll clean up the commit message and push another branch. if you're in a position to test the patch in both opac and staff search, it'd be a much appreciated cross check. |
| 16:00 |
Dyrcona |
miker: Sure thing! |
| 16:59 |
|
mmorgan left #evergreen |
| 17:00 |
Stompro |
Who do I contact to add something to the tabular release notes? |
| 09:59 |
Stompro |
csharp_, thanks, I found them. |
| 09:59 |
csharp_ |
Stompro: happy to help! |
| 10:01 |
Dyrcona |
Maybe B&T are being DOS'd? They've been having issues lately... |
| 10:01 |
jeffdavis |
looks like you can set a longer timeout interval for Content Cafe in opensrf.xml if that would help with testing |
| 10:02 |
jeffdavis |
not sure setting it to 1.2 minutes is a great idea though |
| 10:06 |
Dyrcona |
We're using Bootstrap 4 with Angular? |
| 10:11 |
Stompro |
jeffdavis, I'm fine with the request timing out quickly... but it just seems strange that the request to opac/extras/ac/jacket/small/r/ doesn't return 404 faster. |
| 10:12 |
Stompro |
And it doesn't seem like it registers with the error counter, that is supposed to disable the added content provider after x number of failures. |
| 11:17 |
csharp_ |
oh postgresql 14, why oh why do you care about where VERBOSE belongs in a VACUUM statement |
| 11:17 |
csharp_ |
VACUUM VERBOSE ANALYZE vs VACUUM ANALYZE VERBOSE |
| 11:35 |
Dyrcona |
There's probably a release note on that, but I never noticed. |
| 11:36 |
Dyrcona |
csharp_: Are you using Pg 14 in produciton, or moving data for testing? |
| 11:57 |
Dyrcona |
If I want a purchase order to try again after a FTP error, I just update the status to retry, right? I can't remember if there are other fields to change. |
| 11:58 |
Dyrcona |
The status of acq.edi_message I mean. |
| 12:01 |
Dyrcona |
Oh well, guess it's time to call it a day. |
| 13:48 |
terranm |
It works! Hallelujah! |
| 14:19 |
|
jihpringle joined #evergreen |
| 14:21 |
csharp_ |
berick++ |
| 15:06 |
Stompro |
More content cafe testings... when load a catalog results page with 10 items, I'm seeing 7 successful requests, and 3 requests that eventually time out. The Timeout setting has no effect, because that only works when the agent is waiting for a response. tcpdump shows that the only communication is the initial packet to setup the connection, and then retries being sent. |
| 15:08 |
Stompro |
Makes me think that B&T is limiting connections per second in a very unfriendly way. |
| 15:15 |
|
mantis1 joined #evergreen |
| 17:05 |
|
jvwoolf left #evergreen |
| 14:35 |
jeffdavis |
At Sitka each library has its own OPAC skin. We have some template customizations in git that apply across all skins. We also have some library-specific settings (logo, colors, banner messages, etc.) that we manage with a separate templating system rather than tracking them in our main git repo. |
| 14:36 |
jeffdavis |
So, rather than maintaining 80 different config.tt2 files for 80 different libraries, we have a system that generates and deploys a different config.tt2 for each library based on some settings in a yaml file. |
| 14:37 |
Dyrcona |
Yeah, I'd probably do the same. We don't allow individual customization beyond changing the logo. |
| 14:38 |
Dyrcona |
We did toy with a separate academic skin several years ago, but it never got past testing. |
| 14:39 |
jihpringle |
the majority of our non-logo/colour customizations are for our academic sites |
| 14:40 |
jihpringle |
we recently added it to our docs to make it easier for our libraries to know what can and can't be customized - http://docs.libraries.coop/sitka/_public_catalogue_customizations.html |
| 14:46 |
Dyrcona |
Well, that sounds reasonable. |
| 15:09 |
mmorgan |
We've made a lot of our opac customizations into org unit settings so we don't need to maintain the tt2 files, just update settings in the db. |
| 15:13 |
csharp_ |
yes, squashing is appealing because it feels cleaner, but then you need to revert one of the squashed commits later and you're like "oh yeah, that's why we don't do it like that" |
| 15:13 |
Dyrcona |
Speaking of settings and what not. I'm trying to test something in eg2, and while I can just go there by putting eg2 in the URL, I thought there was a setting somewhere that would do it automagically from the staff opac. |
| 15:13 |
Dyrcona |
csharp_: Exactly that. |
| 15:21 |
Dyrcona |
Hmm... Looks like I'll have to edit nav.component.html |
| 15:26 |
Dyrcona |
So, I'm readying this right that current master still goes to AngularJS for circulation functions from eg2? |
| 16:00 |
mmorgan |
Stompro: Yes, I'm seeing issues in the opac and Novelist today. The service was unavailable back on 11/21, but it restored the next day. |
| 16:02 |
Stompro |
I keep turning it back on, and then having the requests start to fail an hour to a few minutes later. |
| 16:06 |
Dyrcona |
I suspect B&T have issues again. |
| 16:06 |
Dyrcona |
At any rate, I successfully modified nav.component.html to go to eg2 for checkin, checkout, and renewal.....so I can test the changes that I want to make. |
| 16:08 |
* Dyrcona |
should stash that commit in a special branch for later reuse. |
| 16:22 |
Dyrcona |
If I use ng build --watch do I have to specify the output directory? If so, is it /openils/var/web/eg2 or /openils/var/web/eg2/en-US ? |
| 16:23 |
berick |
Dyrcona: ng build puts files into $Evergreen/Open-ILS/web/eg2/ |
| 10:21 |
|
dguarrac joined #evergreen |
| 11:42 |
Dyrcona |
Libraries need to clear their hold shelves. |
| 11:43 |
mmorgan |
:) |
| 11:48 |
Dyrcona |
I'm not having any luck with the patches for Lp 1971745. I guess my local test system's database is just too slow. Neither patch seems to make a difference for me on an "optimized" database or a default configured database with production data. |
| 11:48 |
pinesol |
Launchpad bug 1971745 in Evergreen 3.9 "Holds shelf list can fail to retrieve results" [Undecided,Confirmed] https://launchpad.net/bugs/1971745 |
| 12:02 |
|
jihpringle joined #evergreen |
| 12:02 |
|
Christineb joined #evergreen |
| 12:33 |
Dyrcona |
The patches are working for me on another development system running 3.7.3.... |
| 12:38 |
Dyrcona |
I wonder if the problem is the other db server, or if something went wrong with the upgrade from 3.7.3 to master? |
| 12:38 |
* Dyrcona |
will test on a 3.7.3 vm with the other db server. |
| 12:49 |
Dyrcona |
Hmm... That system with the "bad" db server won't load a hold shelf with about 9 copies. |
| 12:49 |
Dyrcona |
OK. It loaded on the second try. |
| 13:01 |
Dyrcona |
With or without the patches this other test system can't grab a hold shelf with 100 items. |
| 13:01 |
Dyrcona |
@blame the db server |
| 13:01 |
pinesol |
Dyrcona: the db server is the SPY! |
| 13:28 |
Dyrcona |
This is the server that took 5 minutes to connect cstore, prcrud, and storage last week, so.... |
| 15:25 |
Stompro |
Dyrcona, "there's a reason we setup the offline directory twice in our Apache configuration? It's done in eg.conf and again in eg_vhost.conf" I noticed this also, it seemed to be needed when I tried to add a new cgi script, I don't understand why though. |
| 15:25 |
Stompro |
Dyrcona++ thanks for looking at the hold shelf display bug. |
| 15:29 |
Dyrcona |
Stompro: That's interesting about the configuration and the cgi script. I might investigate that. |
| 15:30 |
Dyrcona |
I really wanted to test the hold shelf fixes on Pg 15, but that was on the server that has issues.... |
| 15:38 |
|
jvwoolf left #evergreen |
| 15:41 |
Dyrcona |
Hrm... What version of PostgreSQL should I recommend in the branch that removes Pg 10? I'm inclined to go with Pg 14. |
| 15:41 |
Dyrcona |
Maybe I'll put this branch off until we can have some developer discussion around it. |
| 15:45 |
Stompro |
I don't have any input on that, but I'm looking forward to having a recommendation to follow :-) |
| 15:49 |
Dyrcona |
Well, I'm not sure anyone else will have much to say about that at the next dev. meeting. I'm still inclined to recommend Pg 14, since I've been testing new development on Concerto with that. My latest 2 vms for concerto will use Pg 15 because I used them to develop/test the Pg 15 branch. |
| 15:50 |
berick |
i'm looking at 14 as our next upgrade target fwiw |
| 15:50 |
berick |
and have been using it for dev, etc. |
| 15:50 |
Dyrcona |
I also wonder if we should come up with recommendations for upgrading Pg versions. There are often extra steps required with Evergreen. |
| 15:05 |
JBoyer |
#info sandbegja and others are looking into AngularJS node module security updates |
| 15:05 |
JBoyer |
#link https://bugs.launchpad.net/evergreen/+bug/1992529 |
| 15:05 |
pinesol |
Launchpad bug 1992529 in Evergreen "Upgrade insecure npm dependencies for angularjs staff client" [Medium,New] |
| 15:06 |
JBoyer |
It looks like things are going alright, terranm was able to successfully do some testing but no one involved seems to be here to expand on it. |
| 15:06 |
JBoyer |
There is a branch available that anyone comfortable poking at Angular should take a look at, if it made it in before 3.10 that would be great. |
| 15:07 |
JBoyer |
#topic Evergreen Release Updates |
| 15:07 |
JBoyer |
Any updates from the 3.10 relteam? |
| 15:21 |
gmcharlt |
or "go through all of the TODO and FIXME comments in the code" |
| 15:21 |
berick |
agreed a list of common issues could be helpful |
| 15:21 |
gmcharlt |
or "go through the release notes for the past few years and note any deprecation announcements" |
| 15:24 |
jeffdavis |
Given our code, is it possible to automate something like testing that a shelving location selector can be scoped by org unit? |
| 15:26 |
gmcharlt |
generally speaking, yes - more unit tests for the Angular components is certainly possible |
| 15:26 |
JBoyer |
sandbergja demo'd some "e2e" testing during the hackaway that could potentially do things like that; they're Angular tests that drive the browser and verify its results. |
| 15:26 |
gmcharlt |
but that's a very concrete way of answering the question |
| 15:26 |
gmcharlt |
jeffdavis: is the question more about how to enumerate and document standing expectations for behavior? |
| 15:27 |
jeff |
If we start with a list that is suitable for use as a checklist, then we could use the start with using the checklist and potentially craft some automated lint-like checks of common issues like unscoped selectors. |
| 15:29 |
gmcharlt |
commit message templates embedding a (brief) checklist might also be a way - https://thoughtbot.com/blog/better-commit-messages-with-a-gitmessage-template |
| 15:30 |
jeffdavis |
sorry, I keep typing and deleting responses :) |
| 15:30 |
|
shulabear joined #evergreen |
| 15:30 |
jeffdavis |
a checklist is one example of a way to improve the QA part of our test/commit process, I think we're interested in any kind of solution that improves QA to avoid these kind of recurring problems |
| 15:31 |
jeffdavis |
(it is a very broad problem set for sure, and I agree that formulating a more specific list would be a great next step) |
| 15:32 |
jeffdavis |
I can do some more work with our local support folks to gather more specific examples and flesh out a list for next meeting. |
| 15:33 |
JBoyer |
jeffdavis++ |
| 15:33 |
JBoyer |
Care to #action that for the notes? |
| 15:33 |
shulabear |
jeffdavis++ |
| 16:00 |
JBoyer |
#link https://bugs.launchpad.net/evergreen/+bug/1948693 |
| 16:00 |
pinesol |
Launchpad bug 1948693 in Evergreen "Migrate from NgbTabset to NgbNav (from ng-bootstrap)" [Medium,Confirmed] - Assigned to Stephanie Leary (stephanieleary) |
| 16:01 |
JBoyer |
berick, sorry for the lateness, but if there's anything you'd like to add it's all yours |
| 16:01 |
berick |
thanks |
| 16:01 |
berick |
a couple quick things |
| 16:01 |
berick |
I'm planning to start down the path of updating to Angular 14, Bootstrap 5, etc. |
| 16:02 |
berick |
first part is review/test/merge of bug 1948693, which now has a patch |
| 16:02 |
pinesol |
Launchpad bug 1948693 in Evergreen "Migrate from NgbTabset to NgbNav (from ng-bootstrap)" [Medium,Confirmed] https://launchpad.net/bugs/1948693 - Assigned to Stephanie Leary (stephanieleary) |
| 16:02 |
berick |
it touches lot of UI's, so my question is if I post it on a public VM somewhere can someone help me bounce through and test the UIs? |
| 16:02 |
berick |
basically making sure they load OK and tabs work OK |
| 16:03 |
berick |
i'll post a link to the LP when it's testable |
| 16:03 |
berick |
i don't need a committment, just putting that out there. |
| 16:03 |
sleary |
I would appreciate a lot of eyes on this one. Obviously I ran through them before committing, but there are a few areas where I didn't have adequate test data |
| 16:03 |
JBoyer |
I'm sure pinging the dev lists would help get some people to poke at it too. |
| 16:04 |
berick |
more generally, the Angular etc update is going to affect probably every angular dependency. |
| 16:04 |
berick |
it's not something we'll want to let linger once it's ready |
| 16:12 |
JBoyer |
Also, scottangel feel free to ask any questions you might have now, it's basically a free-for-all when meetings aren't going on. :) |
| 16:12 |
scottangel |
Well then... One question I have is... can I help with the Bootstrap 5 conversion? I love me some bootstraps. |
| 16:13 |
* jeff |
pokes at Google Calendar a bit more... |
| 16:13 |
berick |
miker: if your patch involves updating nodejs or adding new deps, then I'd say yes to committing it |
| 16:14 |
berick |
scottangel++ # we will need lots of manual testing |
| 16:14 |
miker |
well, it /shouldn't/ require that. just a consequence of installing a new version of the node.js binaries |
| 16:14 |
miker |
and rebuilding node_modules |
| 16:14 |
scottangel |
I can help with that as well. |
