| 14:05 |
Dyrcona |
mantis1: If you signoff the branch, all you have to do on the ticket is say that you pushed a signoff branch and where the branch is. |
| 14:05 |
Dyrcona |
It's probably best to wait until after you've tested it. I don't add my signoff if I can't get it to work. |
| 14:05 |
mmorgan |
mantis1: It needs a pullrequest tag, the signoff tag gets added by the tester |
| 14:05 |
Dyrcona |
Oh, yeah. You can add the signoff tag if you tested it. |
| 14:06 |
Dyrcona |
I may have misunderstood the situation. |
| 14:06 |
Dyrcona |
The original author doesn't add the signoff tag for their signoff. |
| 14:06 |
mantis1 |
mmorgan++ Dyrcona++ |
| 15:08 |
Dyrcona |
jihpringle++ jeffdavis++ |
| 15:08 |
terranm |
jihpringle++ jeffdavis++ |
| 15:08 |
Dyrcona |
Some of those will slow things down. |
| 15:08 |
jeffdavis |
Testers could also use it during bug squashing weeks, people doing acceptance testing on paid development project, etc. |
| 15:09 |
sandbergja |
3 and 7 also seem difficult. We don't really have comprehensive information about how all our functionality/settings are supposed to work. |
| 15:09 |
sandbergja |
be it 100% test coverage or a super-detailed manual |
| 15:10 |
JBoyer |
I do like the idea of basing an interface comparison checklist on it. You don't necessarily need to know how to use all of an interface if you can tell that you can get to X on dojo but not Angular. (so long as it's not an intentional workflow change) |
| 15:10 |
sandbergja |
so I'd be interested in some conversation about how developers can get all that info before embarking on an angularization |
| 15:13 |
Dyrcona |
I wonder if the Angular version needs to have feature parity with the previous interfaces(s). What if we came up with better ways of doing things? |
| 15:14 |
mmorgan |
re: library settings - it seems like calls to get library settings in existing code shouldn't be too hard to identify. |
| 15:14 |
JBoyer |
That's what I was getting at yeah. If you can get the same outcome with a different UI that's fine, but being able to do something in the old and busted but not the new hotness isn't great. |
| 15:15 |
tlittle |
#info tlittle = Tiffany Little, PINES |
| 15:17 |
jeffdavis |
IMO the state of our tests/docs means that we'll never catch all of these problems. I think the idea with a checklist was to make sure we're consciously thinking about these types of things at some point during the contribution process. |
| 15:17 |
berick |
yeah, what jeffdavis said |
| 15:17 |
Dyrcona |
This is going to be unpopular, but maybe we need stricter standards than "works for me" and X number of signoffs. |
| 15:18 |
sandbergja |
Dyrcona: what did you have in mind? |
| 15:19 |
berick |
so more emphasis is good |
| 15:19 |
berick |
and having a record of usual gotchas helps focus |
| 15:20 |
abneiman |
this is also where non-technical end users can be helpful - in terms of interface & workflow evalution |
| 15:20 |
Dyrcona |
sandbergja: I'm not really looking for more process, but I do think we need better automated tests, etc. I don't have much specific in mind at the moment. |
| 15:20 |
mmorgan |
abneiman++ |
| 15:21 |
Dyrcona |
Also, what abneiman said. We need more end users involved. |
| 15:21 |
tlittle |
abneiman++ |
| 15:22 |
mmorgan |
It's difficult for end users to get more involved, difficult for them to get their hands on the new development to test on. |
| 15:22 |
JBoyer |
Yeah, testers rarely need to be conversant in the technical bits, and sometimes the tech types don't know so much about how the end users do their thing. |
| 15:23 |
mmorgan |
Bugsquashing week is huge, but test systems are not real world. |
| 15:23 |
gmcharlt |
yeah - I think an additional factor is committer time, and more automation can help around the edges |
| 15:23 |
tlittle |
Numbers 6-7 are concrete things that fall squarely on the dev's shoulders, though, imo |
| 15:24 |
mmorgan |
tlittle++ |
| 15:25 |
JBoyer |
But to mmorgan's point, it's difficult to test on realistic data if you don't have the local staff to build and rebuild test systems. :-/ |
| 15:25 |
Dyrcona |
Well, hire someone. What we need is more resources. It's that simple. |
| 15:26 |
terranm |
Yeah, we find a lot of issues when we do our intensive pre-upgrade testing with a copy of real data. And then more issues when we go live. |
| 15:27 |
mmorgan |
Dyrcona: Agreed more resources, but imo it doesn't seem that simple to hire someone and bring them up the Evergreen learning curve. |
| 15:29 |
berick |
it may be helpful to consider that this phase of EG dev won't last forever. |
| 15:29 |
berick |
there's only so many non-Angular UI's |
| 15:31 |
dmoore |
oh duh, thanks |
| 15:31 |
sleary |
Even just focusing on the Angular UIs is a steep learning curve. I have had trouble with the lack of inline documentation on what various components do. (This topic is on the next new dev agenda, btw.) |
| 15:31 |
csharp_ |
dmoore: feel free to ask if there are other acronyms/jargon you don't understand |
| 15:32 |
sandbergja |
Dyrcona: for me, I think that goes back to better automated tests and docs -- hopefully when/if we migrate away from Angular, we'd have better safeguards against regressions |
| 15:32 |
mmorgan |
All part of the learning curve :) |
| 15:32 |
csharp_ |
part of the answer is new devs - both in general and the new devs group |
| 15:32 |
sandbergja |
also, here's hoping Angular stays healthy for quite some time yet. :-) |
| 15:34 |
csharp_ |
jihpringle++ jeffdavis++ |
| 15:34 |
abneiman |
yes, jihpringle++ jeffdavis++ I have already bookmarked this |
| 15:34 |
terranm |
Same here |
| 15:34 |
sleary |
Very useful! And I would like to do more with automated testing as well. |
| 15:34 |
mmorgan |
Agreed QA gotchas will be useful, there will always be change, we just need to manage it well |
| 15:35 |
mmorgan |
*just* |
| 15:35 |
jvwoolf |
jihpringle++ jeffdavis++ |
| 15:35 |
JBoyer |
So, I do think it should go to the dev lists (perhaps quarterly or so, even) so that we can move on to the second action item from the last meeting. :) |
| 15:35 |
Dyrcona |
This list is a good outline of where we need to pay attention, and tests/standard can be built around it. |
| 15:35 |
JBoyer |
Which is |
| 15:35 |
JBoyer |
#info Bmagic to email the development list about a way to share common Evergreen tools with the community. |
| 15:36 |
JBoyer |
Though I don't think Bmagic is available at the moment and I don't recall seeing this on the lists. We'll kick that can down the road. |
| 15:39 |
JBoyer |
Note though, if anyone else has an interest in the expanded sample data set and opinions on integration and etc. feel free to poke around, |
| 15:40 |
JBoyer |
Ugh, look at me, forgetting to add something to the agenda myself, heh. |
| 15:40 |
JBoyer |
#topic Evergreen Release Updates |
| 15:40 |
JBoyer |
#info I've built a 3.8.2 release to end the 3.8 line now that 3.10 is out, if you can help test it's at https://evergreen-ils.org/downloads/Evergreen-ILS-3.8.2.tar.gz (Note, I'm going to have to make a docs change before it's final-final, but no code changes.) |
| 15:41 |
gmcharlt |
JBoyer++ |
| 15:41 |
shulabear |
Jboyer++ |
| 15:41 |
sandbergja |
JBoyer++ |
| 15:41 |
mmorgan |
JBoyer++ |
| 15:41 |
Dyrcona |
JBoyer++ |
| 15:41 |
JBoyer |
It is 1. wicked late, and 2. not too difficult to test. IF you have access to a 3.8.1 db especially and can help test it (I know the version upgrade is fine for concerto, but you know how that goes) |
| 15:41 |
terranm |
JBoyer++ |
| 15:41 |
tlittle |
JBoyer++ |
| 15:42 |
rfrasur |
JBoyer++ |
| 15:48 |
mmorgan |
jeffdavis: Yes, mostly without branches. A few have pullrequests |
| 15:49 |
jvwoolf |
mmorgan: I've push patches to all of the ones that were showstoppers for us so far |
| 15:49 |
terranm |
We're going ahead with upgrading because they're not show stoppers for us, but we know there will be some grumblings |
| 15:49 |
jvwoolf |
I'm likely done with my concentrated effort unless something comes up when we widen our testing to end users, or after we upgrade |
| 15:50 |
mmorgan |
Template issues, silent failure issues are big ones for us. |
| 15:51 |
mmorgan |
We're still sorting through and prioritizing |
| 15:51 |
|
smorrison joined #evergreen |
| 10:10 |
Dyrcona |
This JabberClient instance is no longer connected to the server |
| 10:13 |
Dyrcona |
ejabberd.log is, of course, useless. |
| 10:26 |
miker |
Dyrcona: I have ideas for making facets always faster (or at least never slower) since they can be turned into unique int IDs, but tuits... |
| 10:29 |
Dyrcona |
miker: Sounds cool. I also think that should be a separate bug from the ones I'm testing. It's just something that I've noticed. |
| 10:30 |
Dyrcona |
It's not always slower, just sometimes. |
| 10:30 |
* Dyrcona |
goes fishing through logs for a really crazy search to test. |
| 10:45 |
Dyrcona |
So, that jabber thing that I reported earlier: Has anyone ever seen services crash and have 8 listeners all using 100% cpu? This appears to have happened right after I restarted services this morning. |
| 10:47 |
Dyrcona |
I suspect something in the configuration must be busted. |
| 10:57 |
Dyrcona |
Huh. Just restarting ejabberd seems to have fixed it for now. |
| 12:41 |
jvwoolf |
(We don't have symspell in our current version, still on 3.6.5) |
| 12:49 |
pinesol |
News from commits: Docs: updating Global Flags docs <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=8326611f8dc232dda993dca412e429713189999a> |
| 13:49 |
csharp_ |
jvwoolf: it's definitely quicker without all the triggers if you can spare the downtime |
| 13:54 |
jvwoolf |
csharp_++ |
| 13:55 |
jvwoolf |
We have some time so I might test still test with triggers and without |
| 13:56 |
jvwoolf |
My guess is that the libraries may prefer to be offline than to have their Sunday staff let loose on a new version that might be slow because of the reingest, but I'll ask that question to our user group |
| 14:05 |
Dyrcona |
I think there are some patches in working branches that can make the ingest more palatable, but it looks like I have it on my other laptop, unless the one I was looking at went in. |
| 14:07 |
jvwoolf |
Dyrcona: The fix for bug 1931737 is in 3.9.1 |
| 14:07 |
pinesol |
Launchpad bug 1931737 in Evergreen 3.8 "Did you mean breaks parallel reingest and causes deadlocks when loading/overlaying bib records in the client" [High,Fix committed] https://launchpad.net/bugs/1931737 |
| 11:27 |
berick |
BTW, not sure if the backend will cache anything for a search that times out |
| 11:28 |
Dyrcona |
Well, it seems to work in the OPAC most of the time. |
| 11:29 |
Dyrcona |
I'll have to compare the two in more depth later. |
| 11:29 |
* Dyrcona |
is testing some search fixes on production data. |
| 11:31 |
Dyrcona |
It might be this particular search is way too broad given our data, but this has been consistent with other searches, IIRC. |
| 11:32 |
Dyrcona |
This search isn't returning results in the OPAC, either, and it gives up almost immediately. |
| 11:33 |
Dyrcona |
The query doesn't run nearly as long in the database now, either. |
| 11:56 |
|
jvwoolf joined #evergreen |
| 12:06 |
miker |
not here, but staff search never uses the cache. else newly added records wouldn't show up |
| 12:16 |
miker |
in the olden days, the #staff modifier would force skipping the cache. I think that still works? but docache probably overrides that heuristic |
| 12:20 |
Dyrcona |
Well, I've done some more fiddling around with that bug and comparing produciton with my test system with search patches. I get results in production regardless of filters, etc. On the test system, I either have to change to a title search or limit by library to get results. |
| 12:20 |
Dyrcona |
I suppose it is just my test database being slower than production. |
| 12:26 |
|
dguarrac joined #evergreen |
| 12:34 |
|
collum joined #evergreen |
| 12:47 |
|
collum joined #evergreen |
| 10:38 |
Dyrcona |
Not bad though, 3.7 seconds for a seq scan on 3.7 million records. |
| 10:39 |
Bmagic |
better than a human |
| 10:43 |
Dyrcona |
Adding an index on bre.vis_attr_vector doesn't make a difference in the explain output for the simple query. |
| 10:44 |
Dyrcona |
Also, there's an error in the SQL I pasted before. I'm actually running this to test: select id from biblio.record_entry where vis_attr_vector IS NULL OR NOT int4range(0,268435455,'[]') @> ANY(vis_attr_vector); |
| 10:44 |
Dyrcona |
I'm going to add an index on acvac.vis_attr_vector and try the big query again. |
| 10:47 |
Dyrcona |
That doesn't seem to make a difference either, so I'll try the procedural index that miker suggested yesterday next. |
| 10:54 |
Dyrcona |
"create index bre_not_deleted_idx ON biblio.record_entry (id) where deleted = 'f';" doesn't seem to help, either. |
| 11:14 |
miker |
select * from config.internal_flag where name like '%delete%'; if you're in psql |
| 11:16 |
Dyrcona |
I'm pretty sure that's on, but I'll double check. |
| 11:17 |
Dyrcona |
Nope. enabled = 'f'. |
| 11:17 |
miker |
kk |
| 11:20 |
miker |
Dyrcona: a big change in pg 12 is the default for non-materialized CTEs. we can't just do it across the board yet, but do you have the full original query handy to test by hand? if so, in front of each CTE (WITH-clause), after the AS keyword, what happens if you add MATERIALIZED between AS and the open-( ? |
| 11:21 |
Dyrcona |
I'll give that a shot. I have it open in my editor. |
| 11:23 |
miker |
if that makes things happy -- and I can imagine it will because estimates of the number rows coming out of a tsearch CTE are ... usually way off -- then we /can/ add a PG version check and add it when it's available. there's precedent for that in the record attribute testing infrastructure. |
| 11:27 |
Dyrcona |
It's worse, but I may have missed a couple: Execution Time: 115367.485 ms |
| 11:31 |
Dyrcona |
OK! I missed c_attr and b_attr before. after adding materialized to those two, it was much better: Execution Time: 3123.997 ms |
| 11:32 |
Dyrcona |
My guess is c_attr as materialized does the magic. |
| 11:34 |
Dyrcona |
"That's a bingo!" |
| 11:37 |
miker |
limit and offset are usually optimization fences ... Dyrcona, would you mind trying this? remove the MATERIALIZED on c_attr and add "OFFSET 0" just before the closing ) on that CTE |
| 11:38 |
Dyrcona |
OK. I was about to say that we'd have to bump the minimum Pg version to 12 if we use materialized CTEs. |
| 11:38 |
miker |
if that has the same effect, we should be able to (for now, until PG learns how to ignore useless limit/offset) do that. |
| 11:39 |
miker |
as for versioning, we already test for the version in one other place and change query structure based on that. we could do it here if MATERIALIZED is the answer |
| 11:40 |
miker |
we could also mark the function volatile to force materialization |
| 11:40 |
Dyrcona |
Execution Time: 2675.806 ms |
| 11:40 |
miker |
oh hey, look at that |
| 11:40 |
Dyrcona |
So, the offset works, too. |
| 11:44 |
Dyrcona |
Should I try that, too, or are we happy with "offset 0"? |
| 11:44 |
miker |
it is, but only in OPAC searches. staff search does use search.calculate_visibility_attribute_test thought |
| 11:45 |
miker |
though, even |
| 11:45 |
Dyrcona |
FYI, I'm testing on Pg 13, but it probably doesn't matter. |
| 11:47 |
miker |
offset 0 is the more self-documenting variant that works for all pg version, for now. though, again, we can version-test and use materialized in 12+ |
| 11:49 |
miker |
testing offset 0 on an older version, just to be sure |
| 11:49 |
Dyrcona |
I was talking about marking search.calculate_visibility_attribute_test as volatile. |
| 11:50 |
miker |
right |
| 11:50 |
Dyrcona |
I'm going to try it just because I can. |
| 11:50 |
miker |
+1, try away! |
| 11:52 |
Dyrcona |
Volatile without offset 0 works, too, but I'll defer to your opinion on offset 0. |
| 11:53 |
Dyrcona |
Execution time is about the same, 4ms difference which is probably not related to the change. |
| 12:17 |
miker |
I got curious and checked the difference between ts_rank and ts_rank_cd ... it's bigger than I thought :( |
| 12:18 |
miker |
in my test query, 3ms vs 90ms, respectively |
| 12:22 |
|
jihpringle joined #evergreen |
| 12:28 |
miker |
Dyrcona: do you want to branchify that, or shall I? |
| 12:28 |
Dyrcona |
miker: ts_rank makes a roughly 67ms difference in execution time. |
| 12:28 |
Dyrcona |
in my test environment. |
| 12:29 |
Dyrcona |
If you want to branch it, that's fine with me. I was going to open a Lp bug, but I should actually look into yet another OOM on the db server. |
| 12:29 |
Dyrcona |
It happened just before noon. |
| 12:29 |
miker |
oh, well, that's not enough to warrant further investigation this moment. my test case may be unusual |
| 12:29 |
miker |
I'll put together the patch and await your LP, then? |
| 12:30 |
Dyrcona |
OK with me. |
| 12:36 |
Dyrcona |
Is it possible to POST an OPAC search? |
| 12:42 |
miker |
it is |
| 12:43 |
miker |
(and, note, i was incorrect about staff search not using the patron vis test function -- it does, it just ignores that the the WHERE clause) |
| 12:43 |
miker |
Dyrcona: https://git.evergreen-ils.org/?p=working/Evergreen.git;a=shortlog;h=refs/heads/user/miker/optimization-fence-for-pg-12-CTEs-in-search for your eyeballs |
| 12:45 |
Dyrcona |
miker++ |
| 12:46 |
Dyrcona |
Re POST: I guess we'll need to work out a mitigation for POSTS with "bad" queries in them. |
| 13:33 |
|
kworstell-isl joined #evergreen |
| 15:21 |
jihpringle |
Bmagic: depends on the version, there's an admin interface starting in 3.8 or 3.9 I think (we definitely have it in 3.9) |
| 15:21 |
Bmagic |
jihpringle++ |
| 15:22 |
mmorgan |
It's new in 3.9 |
| 15:22 |
Stompro |
I testing on 3.9, so I get to try that out. |
| 15:25 |
jeffdavis |
In addition to the new staff portal config options, we add an iframe to the page so we can display news updates from another source: http://git.sitka.bclibraries.ca/gitweb/?p=sitka/evergreen.git;a=commitdiff;h=45bfec6b5b1a7a8d0dfd898aa68926d17bb8f1ca |
| 15:25 |
jeffdavis |
I should probably turn that into a feature request. |
| 15:28 |
Stompro |
jeffdavis, thanks, that is interesting. |
| 15:31 |
|
kworstell-isl joined #evergreen |
| 15:34 |
Stompro |
Shoot, "it won’t try to merge branch, system, and consortial-level entries" I cannot have System portal with the odd entry for a specific branch. But I get that would be more complex to figure out. |
| 15:53 |
jihpringle |
ya, if you want different portals for different org units with slight differences you have to re-create the portal for each org unit you want (but there is a clone button) |
| 15:58 |
miker |
Dyrcona: I'll clean up the commit message and push another branch. if you're in a position to test the patch in both opac and staff search, it'd be a much appreciated cross check. |
| 16:00 |
Dyrcona |
miker: Sure thing! |
| 16:59 |
|
mmorgan left #evergreen |
| 17:00 |
Stompro |
Who do I contact to add something to the tabular release notes? |
| 09:59 |
Stompro |
csharp_, thanks, I found them. |
| 09:59 |
csharp_ |
Stompro: happy to help! |
| 10:01 |
Dyrcona |
Maybe B&T are being DOS'd? They've been having issues lately... |
| 10:01 |
jeffdavis |
looks like you can set a longer timeout interval for Content Cafe in opensrf.xml if that would help with testing |
| 10:02 |
jeffdavis |
not sure setting it to 1.2 minutes is a great idea though |
| 10:06 |
Dyrcona |
We're using Bootstrap 4 with Angular? |
| 10:11 |
Stompro |
jeffdavis, I'm fine with the request timing out quickly... but it just seems strange that the request to opac/extras/ac/jacket/small/r/ doesn't return 404 faster. |
| 10:12 |
Stompro |
And it doesn't seem like it registers with the error counter, that is supposed to disable the added content provider after x number of failures. |
| 11:17 |
csharp_ |
oh postgresql 14, why oh why do you care about where VERBOSE belongs in a VACUUM statement |
| 11:17 |
csharp_ |
VACUUM VERBOSE ANALYZE vs VACUUM ANALYZE VERBOSE |
| 11:35 |
Dyrcona |
There's probably a release note on that, but I never noticed. |
| 11:36 |
Dyrcona |
csharp_: Are you using Pg 14 in produciton, or moving data for testing? |
| 11:57 |
Dyrcona |
If I want a purchase order to try again after a FTP error, I just update the status to retry, right? I can't remember if there are other fields to change. |
| 11:58 |
Dyrcona |
The status of acq.edi_message I mean. |
| 12:01 |
Dyrcona |
Oh well, guess it's time to call it a day. |
| 13:48 |
terranm |
It works! Hallelujah! |
| 14:19 |
|
jihpringle joined #evergreen |
| 14:21 |
csharp_ |
berick++ |
| 15:06 |
Stompro |
More content cafe testings... when load a catalog results page with 10 items, I'm seeing 7 successful requests, and 3 requests that eventually time out. The Timeout setting has no effect, because that only works when the agent is waiting for a response. tcpdump shows that the only communication is the initial packet to setup the connection, and then retries being sent. |
| 15:08 |
Stompro |
Makes me think that B&T is limiting connections per second in a very unfriendly way. |
| 15:15 |
|
mantis1 joined #evergreen |
| 17:05 |
|
jvwoolf left #evergreen |
| 14:35 |
jeffdavis |
At Sitka each library has its own OPAC skin. We have some template customizations in git that apply across all skins. We also have some library-specific settings (logo, colors, banner messages, etc.) that we manage with a separate templating system rather than tracking them in our main git repo. |
| 14:36 |
jeffdavis |
So, rather than maintaining 80 different config.tt2 files for 80 different libraries, we have a system that generates and deploys a different config.tt2 for each library based on some settings in a yaml file. |
| 14:37 |
Dyrcona |
Yeah, I'd probably do the same. We don't allow individual customization beyond changing the logo. |
| 14:38 |
Dyrcona |
We did toy with a separate academic skin several years ago, but it never got past testing. |
| 14:39 |
jihpringle |
the majority of our non-logo/colour customizations are for our academic sites |
| 14:40 |
jihpringle |
we recently added it to our docs to make it easier for our libraries to know what can and can't be customized - http://docs.libraries.coop/sitka/_public_catalogue_customizations.html |
| 14:46 |
Dyrcona |
Well, that sounds reasonable. |
| 15:09 |
mmorgan |
We've made a lot of our opac customizations into org unit settings so we don't need to maintain the tt2 files, just update settings in the db. |
| 15:13 |
csharp_ |
yes, squashing is appealing because it feels cleaner, but then you need to revert one of the squashed commits later and you're like "oh yeah, that's why we don't do it like that" |
| 15:13 |
Dyrcona |
Speaking of settings and what not. I'm trying to test something in eg2, and while I can just go there by putting eg2 in the URL, I thought there was a setting somewhere that would do it automagically from the staff opac. |
| 15:13 |
Dyrcona |
csharp_: Exactly that. |
| 15:21 |
Dyrcona |
Hmm... Looks like I'll have to edit nav.component.html |
| 15:26 |
Dyrcona |
So, I'm readying this right that current master still goes to AngularJS for circulation functions from eg2? |
| 16:00 |
mmorgan |
Stompro: Yes, I'm seeing issues in the opac and Novelist today. The service was unavailable back on 11/21, but it restored the next day. |
| 16:02 |
Stompro |
I keep turning it back on, and then having the requests start to fail an hour to a few minutes later. |
| 16:06 |
Dyrcona |
I suspect B&T have issues again. |
| 16:06 |
Dyrcona |
At any rate, I successfully modified nav.component.html to go to eg2 for checkin, checkout, and renewal.....so I can test the changes that I want to make. |
| 16:08 |
* Dyrcona |
should stash that commit in a special branch for later reuse. |
| 16:22 |
Dyrcona |
If I use ng build --watch do I have to specify the output directory? If so, is it /openils/var/web/eg2 or /openils/var/web/eg2/en-US ? |
| 16:23 |
berick |
Dyrcona: ng build puts files into $Evergreen/Open-ILS/web/eg2/ |
| 10:21 |
|
dguarrac joined #evergreen |
| 11:42 |
Dyrcona |
Libraries need to clear their hold shelves. |
| 11:43 |
mmorgan |
:) |
| 11:48 |
Dyrcona |
I'm not having any luck with the patches for Lp 1971745. I guess my local test system's database is just too slow. Neither patch seems to make a difference for me on an "optimized" database or a default configured database with production data. |
| 11:48 |
pinesol |
Launchpad bug 1971745 in Evergreen 3.9 "Holds shelf list can fail to retrieve results" [Undecided,Confirmed] https://launchpad.net/bugs/1971745 |
| 12:02 |
|
jihpringle joined #evergreen |
| 12:02 |
|
Christineb joined #evergreen |
| 12:33 |
Dyrcona |
The patches are working for me on another development system running 3.7.3.... |
| 12:38 |
Dyrcona |
I wonder if the problem is the other db server, or if something went wrong with the upgrade from 3.7.3 to master? |
| 12:38 |
* Dyrcona |
will test on a 3.7.3 vm with the other db server. |
| 12:49 |
Dyrcona |
Hmm... That system with the "bad" db server won't load a hold shelf with about 9 copies. |
| 12:49 |
Dyrcona |
OK. It loaded on the second try. |
| 13:01 |
Dyrcona |
With or without the patches this other test system can't grab a hold shelf with 100 items. |
| 13:01 |
Dyrcona |
@blame the db server |
| 13:01 |
pinesol |
Dyrcona: the db server is the SPY! |
| 13:28 |
Dyrcona |
This is the server that took 5 minutes to connect cstore, prcrud, and storage last week, so.... |
| 15:25 |
Stompro |
Dyrcona, "there's a reason we setup the offline directory twice in our Apache configuration? It's done in eg.conf and again in eg_vhost.conf" I noticed this also, it seemed to be needed when I tried to add a new cgi script, I don't understand why though. |
| 15:25 |
Stompro |
Dyrcona++ thanks for looking at the hold shelf display bug. |
| 15:29 |
Dyrcona |
Stompro: That's interesting about the configuration and the cgi script. I might investigate that. |
| 15:30 |
Dyrcona |
I really wanted to test the hold shelf fixes on Pg 15, but that was on the server that has issues.... |
| 15:38 |
|
jvwoolf left #evergreen |
| 15:41 |
Dyrcona |
Hrm... What version of PostgreSQL should I recommend in the branch that removes Pg 10? I'm inclined to go with Pg 14. |
| 15:41 |
Dyrcona |
Maybe I'll put this branch off until we can have some developer discussion around it. |
| 15:45 |
Stompro |
I don't have any input on that, but I'm looking forward to having a recommendation to follow :-) |
| 15:49 |
Dyrcona |
Well, I'm not sure anyone else will have much to say about that at the next dev. meeting. I'm still inclined to recommend Pg 14, since I've been testing new development on Concerto with that. My latest 2 vms for concerto will use Pg 15 because I used them to develop/test the Pg 15 branch. |
| 15:50 |
berick |
i'm looking at 14 as our next upgrade target fwiw |
| 15:50 |
berick |
and have been using it for dev, etc. |
| 15:50 |
Dyrcona |
I also wonder if we should come up with recommendations for upgrading Pg versions. There are often extra steps required with Evergreen. |
| 15:05 |
JBoyer |
#info sandbegja and others are looking into AngularJS node module security updates |
| 15:05 |
JBoyer |
#link https://bugs.launchpad.net/evergreen/+bug/1992529 |
| 15:05 |
pinesol |
Launchpad bug 1992529 in Evergreen "Upgrade insecure npm dependencies for angularjs staff client" [Medium,New] |
| 15:06 |
JBoyer |
It looks like things are going alright, terranm was able to successfully do some testing but no one involved seems to be here to expand on it. |
| 15:06 |
JBoyer |
There is a branch available that anyone comfortable poking at Angular should take a look at, if it made it in before 3.10 that would be great. |
| 15:07 |
JBoyer |
#topic Evergreen Release Updates |
| 15:07 |
JBoyer |
Any updates from the 3.10 relteam? |
| 15:21 |
gmcharlt |
or "go through all of the TODO and FIXME comments in the code" |
| 15:21 |
berick |
agreed a list of common issues could be helpful |
| 15:21 |
gmcharlt |
or "go through the release notes for the past few years and note any deprecation announcements" |
| 15:24 |
jeffdavis |
Given our code, is it possible to automate something like testing that a shelving location selector can be scoped by org unit? |
| 15:26 |
gmcharlt |
generally speaking, yes - more unit tests for the Angular components is certainly possible |
| 15:26 |
JBoyer |
sandbergja demo'd some "e2e" testing during the hackaway that could potentially do things like that; they're Angular tests that drive the browser and verify its results. |
| 15:26 |
gmcharlt |
but that's a very concrete way of answering the question |
| 15:26 |
gmcharlt |
jeffdavis: is the question more about how to enumerate and document standing expectations for behavior? |
| 15:27 |
jeff |
If we start with a list that is suitable for use as a checklist, then we could use the start with using the checklist and potentially craft some automated lint-like checks of common issues like unscoped selectors. |
| 15:29 |
gmcharlt |
commit message templates embedding a (brief) checklist might also be a way - https://thoughtbot.com/blog/better-commit-messages-with-a-gitmessage-template |
| 15:30 |
jeffdavis |
sorry, I keep typing and deleting responses :) |
| 15:30 |
|
shulabear joined #evergreen |
| 15:30 |
jeffdavis |
a checklist is one example of a way to improve the QA part of our test/commit process, I think we're interested in any kind of solution that improves QA to avoid these kind of recurring problems |
| 15:31 |
jeffdavis |
(it is a very broad problem set for sure, and I agree that formulating a more specific list would be a great next step) |
| 15:32 |
jeffdavis |
I can do some more work with our local support folks to gather more specific examples and flesh out a list for next meeting. |
| 15:33 |
JBoyer |
jeffdavis++ |
| 15:33 |
JBoyer |
Care to #action that for the notes? |
| 15:33 |
shulabear |
jeffdavis++ |
| 16:00 |
JBoyer |
#link https://bugs.launchpad.net/evergreen/+bug/1948693 |
| 16:00 |
pinesol |
Launchpad bug 1948693 in Evergreen "Migrate from NgbTabset to NgbNav (from ng-bootstrap)" [Medium,Confirmed] - Assigned to Stephanie Leary (stephanieleary) |
| 16:01 |
JBoyer |
berick, sorry for the lateness, but if there's anything you'd like to add it's all yours |
| 16:01 |
berick |
thanks |
| 16:01 |
berick |
a couple quick things |
| 16:01 |
berick |
I'm planning to start down the path of updating to Angular 14, Bootstrap 5, etc. |
| 16:02 |
berick |
first part is review/test/merge of bug 1948693, which now has a patch |
| 16:02 |
pinesol |
Launchpad bug 1948693 in Evergreen "Migrate from NgbTabset to NgbNav (from ng-bootstrap)" [Medium,Confirmed] https://launchpad.net/bugs/1948693 - Assigned to Stephanie Leary (stephanieleary) |
| 16:02 |
berick |
it touches lot of UI's, so my question is if I post it on a public VM somewhere can someone help me bounce through and test the UIs? |
| 16:02 |
berick |
basically making sure they load OK and tabs work OK |
| 16:03 |
berick |
i'll post a link to the LP when it's testable |
| 16:03 |
berick |
i don't need a committment, just putting that out there. |
| 16:03 |
sleary |
I would appreciate a lot of eyes on this one. Obviously I ran through them before committing, but there are a few areas where I didn't have adequate test data |
| 16:03 |
JBoyer |
I'm sure pinging the dev lists would help get some people to poke at it too. |
| 16:04 |
berick |
more generally, the Angular etc update is going to affect probably every angular dependency. |
| 16:04 |
berick |
it's not something we'll want to let linger once it's ready |
| 16:12 |
JBoyer |
Also, scottangel feel free to ask any questions you might have now, it's basically a free-for-all when meetings aren't going on. :) |
| 16:12 |
scottangel |
Well then... One question I have is... can I help with the Bootstrap 5 conversion? I love me some bootstraps. |
| 16:13 |
* jeff |
pokes at Google Calendar a bit more... |
| 16:13 |
berick |
miker: if your patch involves updating nodejs or adding new deps, then I'd say yes to committing it |
| 16:14 |
berick |
scottangel++ # we will need lots of manual testing |
| 16:14 |
miker |
well, it /shouldn't/ require that. just a consequence of installing a new version of the node.js binaries |
| 16:14 |
miker |
and rebuilding node_modules |
| 16:14 |
scottangel |
I can help with that as well. |
| 09:18 |
|
stephengwills joined #evergreen |
| 09:18 |
Dyrcona |
miker: Since it would take a day pretty much to restore and "re-upgrade" the database, I'll stick with what I've got checked out. I'll see about pulling the top commits from your other branch. |
| 09:57 |
Dyrcona |
miker: Everything is failing with the changes to metabib.reingest_metabib_field_entries. I'm going to back it out. |
| 09:59 |
miker |
awesome. if you have more info than that, that'd be great. I'm not seeing everything fail... |
| 09:59 |
miker |
thanks for testing, regardless |
| 10:02 |
miker |
Dyrcona: that function has changes that depend on the deadlock branch, fwiw |
| 10:03 |
Dyrcona |
Yeah, I have those. I did the DYM deadlock upgrade, too. |
| 10:03 |
Dyrcona |
miker: Here's a PostgreSQL log entry, https://pastebin.com/SsECj4QB |
| 10:13 |
Dyrcona |
I stopped the coordinator, replaced the function, and it seems to be working again. |
| 11:15 |
|
mantis1 joined #evergreen |
| 11:27 |
Dyrcona |
miker: By "replaced the function," I mean that I went back to the version from the base schema. |
| 11:33 |
miker |
ah, ok. well, I confess that the error message in the log doesn't make much sense to me, it's complaining about a different table... but I'll see if I can sus out what's up there |
| 11:34 |
Dyrcona |
I'll try the new implementation again tomorrow or next week. I'm also trying to test a load of records that say they're MARC8, but I have my suspicions. |
| 11:35 |
Dyrcona |
I have some smaller batches that I'll use with the new implementation. |
| 11:36 |
Dyrcona |
Yeah, I wasn't really sure about the error message, either. I've got a couple of thousand more if you want to extract another. |
| 11:36 |
Dyrcona |
Something like 2,500 failed by the time that I noticed. |
| 11:40 |
Dyrcona |
Let me make sure it does the new thing for the password. |
| 11:41 |
Dyrcona |
It does! How about that! |
| 11:41 |
* Dyrcona |
cranks up the Iron Maiden... |
| 11:46 |
miker |
Dyrcona: so, I found the issue. the problem is in the details of PG's fkey checks. the cause is updating the id (which the named table points to as "entry"), even with the original value. we can replace sort_value, since it's the same and (basically) a key as well with the same effect and not bonk into that issue. I'm testing that change now (no explosions after 5min+), but it comes down to this if you want to use "\ef metabib. |
| 11:46 |
miker |
reingest_metabib_field_entries" to adjust the new version in situ: https://pastebin.com/PKDrhfSX |
| 11:46 |
|
jihpringle joined #evergreen |
| 11:48 |
Dyrcona |
miker: Cool, I'll give that a shot after I play with my new SIP accounts. |
| 11:48 |
miker |
+1 |
| 12:15 |
Dyrcona |
I suppose that I could just replace the function while it's running and see what happens. |
| 12:39 |
Dyrcona |
miker++ I swapped ou the function definition and it's still working. |
| 12:39 |
miker |
huzzah! |
| 12:49 |
Dyrcona |
My SIP test is not going so well. I can't tell if I'm failing to log in or if my test script is just blowing up. I'm getting a socket error. |
| 13:04 |
Dyrcona |
hmm. I get no response from server when I try to login with a newly created account, but pre-existing ones work. I even tried running actor.change_password() with the password that I'm using. |
| 13:09 |
Dyrcona |
There's a bug, I think. I was trying to use the location to add a workstation for the new users. When I removed the location, it worked. |
| 13:09 |
|
mrtnnbr joined #evergreen |
| 14:12 |
|
sleary joined #evergreen |
| 14:13 |
Dyrcona |
The other 3 are also deadlocks. |
| 15:29 |
Dyrcona |
So, I am now getting sharelock violations. That's it. I'll have to reload everything next week and start over with a clean upgrade to master, etc., to make sure I have everything, but I am pretty sure that everything is up to date. |
| 15:37 |
jeffdavis |
Do we have a way to flag bugs that need discussion rather than testing for bug squashing week? There's a fix for bug 1863387 that I'd like to commit as-is, but it would be good to know if end users think it needs more work. |
| 15:37 |
pinesol |
Launchpad bug 1863387 in Evergreen "Carousels: shelving location selector should limit initial set of choices" [Medium,Confirmed] https://launchpad.net/bugs/1863387 |
| 15:37 |
Dyrcona |
jeffdavis: needsdiscussion tag. |
| 15:40 |
jeffdavis |
oh, are needsdiscussion bugs already included in bug squashing? that would be great! |
| 15:49 |
sleary |
I forgot about that tag |
| 15:50 |
Dyrcona |
On the plus side, the ingest coordinator has stayed up and running. |
| 15:51 |
|
jvwoolf left #evergreen |
| 16:16 |
mmorgan |
jeffdavis: I am pretty sure the main focus of bug squashing week is bugs with pullrequests, terranm does a masterful job of organizing them on test servers with signoffs as a goal. |
| 16:17 |
mmorgan |
But I would think BSW is a great opportunity to look at needsdiscussion bugs, too. |
| 16:50 |
|
stephengwills left #evergreen |
| 17:06 |
|
mmorgan left #evergreen |
| 18:25 |
|
sandbergja joined #evergreen |
| 18:57 |
pinesol |
News from commits: LP1952931 release notes <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=b33f1170050401a0c6066452a09d391fd97a7cd2> |
| 18:57 |
pinesol |
News from commits: LP1952931 stamp upgrade script <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=25f23ca394d5759f685f6b54933e9d0db18423cd> |
| 18:57 |
pinesol |
News from commits: LP1952931 Support ACQ Advanced Shipment Notices (DESADV -- Dispatch Advice Messages) <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=eae6c7e0eacc9ebf4bfd2014d9c6485a61de8a0c> |
| 19:28 |
gmcharlt |
testing, then pusing fix to 1344 |
| 19:38 |
|
sandbergja joined #evergreen |
| 19:38 |
sandbergja |
gmcharlt++ |
| 19:57 |
pinesol |
News from commits: LP#1724032: add release note entry <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=2cad20e67af7d630b18834313960e00414dd758e> |
| 20:28 |
pinesol |
News from commits: LP1904036 Mark Damaged always handles checkin <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=55941c65290a23c682746c230b96fff12d38efc4> |
| 20:28 |
pinesol |
News from commits: LP1904036 Checkin grid routeTo visible by default <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=5bc6f2816ba2124198c9763f0d17c9aac88b05c4> |
| 20:44 |
* gmcharlt |
claims 1347 |
| 20:57 |
pinesol |
News from commits: LP#1934162: stamp DB update <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=00d6528742c31a7b04aa876e6e4a471fec8c6bfc> |
| 20:57 |
pinesol |
News from commits: LP#1934162: add release note <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=1ea534594a176b72b404f12f8071faf870e96c8f> |
| 20:57 |
pinesol |
News from commits: LP1934162: (follow-up) sync whitespace <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=5ec73fa03414a59bd8516e571da6070b45e3b3ce> |
| 20:57 |
pinesol |
News from commits: LP1934162: add pgtap test, refreshing upgrade script <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=a51d2366a735903adf04cfe7a32af1565313b334> |
| 20:57 |
pinesol |
News from commits: LP#1934162: delete user messages and curbside notes <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=d24c4dbe1e9b3b4dc924647c4c29082242b37622> |
| 21:07 |
* gmcharlt |
claims 1348 |
| 21:16 |
* gmcharlt |
claims 1349 |
| 21:27 |
pinesol |
News from commits: LP#1982031: add release note <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=244f0c732b9358e8c272364047e03f1b92406d06> |
| 00:21 |
pinesol |
News from commits: LP1411819 stamp upgrade script and add release notes <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=8da533ecb4126fdaa6cd6386a4468fe11550e4da> |
| 00:21 |
pinesol |
News from commits: LP1411819 follow-up: add a pgtap test <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=fd0a11c18fbac315b81d04da359ba062d002d916> |
| 00:21 |
pinesol |
News from commits: LP#1411819: org setting to override PATRON_EXCEEDS_FINES penalty on renewals <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=98ce2c013c295509d530df40bed947279dc3b9f5> |
| 00:51 |
pinesol |
News from commits: LP1953692: release notes <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=18c15f0957fdea5d1213ed137d80f879b9aa7ade> |
| 00:51 |
pinesol |
News from commits: LP1953692 follow-up: missing character <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=6b9a7d9bdf64144b8138025713575d9f79c61650> |
| 00:51 |
pinesol |
News from commits: LP1953692 Angular Catalog Record Summary links should open in new tab <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=781f399afa75a6f9fae4ef164c797f1231821f22> |
| 02:18 |
|
tsadok joined #evergreen |
| 07:42 |
|
collum joined #evergreen |
| 08:07 |
|
BDorsey joined #evergreen |
| 09:28 |
|
Dyrcona joined #evergreen |
| 09:45 |
|
dguarrac joined #evergreen |
| 09:46 |
|
jvwoolf joined #evergreen |
| 09:57 |
mmorgan |
Asked this in the hackaway, but I'll try here, too. |
| 09:57 |
mmorgan |
Trying to build a test vm using ansible and I'm getting this: "Open-ILS/src/extras/Makefile.install:156: recipe for target 'ubuntu-bionic-developer' failed" |
| 09:57 |
mmorgan |
What am I missing? |
| 10:00 |
|
sleary joined #evergreen |
| 10:03 |
Dyrcona |
miker: Getting different behavior with ingest_ctl today after the patch from yesterday. I'm loading the same records that I used last week after a database reload. Updating 378 records worked fine. Inserting 314 has produced 7 errors and after that, it seems to have really slowed down. |
| 10:05 |
Dyrcona |
And, of course, after I mention it here, it picks back up again. |
| 10:14 |
Dyrcona |
I have all of the patches from DYM and queued ingest. |
| 10:17 |
Dyrcona |
The rest are all pretty much the same. |
| 10:24 |
Dyrcona |
I processed 1 by hand and it worked. I update fail_time to null and the rest of them worked in parallel. |
| 10:28 |
JBoyer |
mmorgan, regardless of the specific error you'll want to update your testing VM to a later version of Ubuntu; building Evergreen on Bionic from a fresh git checkout is likely to start failing within days. |
| 10:28 |
JBoyer |
see lp 1990969 |
| 10:28 |
pinesol |
Launchpad bug 1990969 in Evergreen "Discontinue Support for Ubuntu 18.04 "Bionic Beaver"" [Wishlist,Confirmed] https://launchpad.net/bugs/1990969 |
| 10:29 |
JBoyer |
OR, if you're running a later version of Ubuntu on your VM it's entirely possible that some package name has changed since bionic and that's what's failing. |
| 11:59 |
Dyrcona |
miker: I noticed two entries for the same author name back to back, so there's some evidence of that. |
| 12:02 |
mmorgan |
Bmagic: Right. Will look at that when I find some time. |
| 12:14 |
|
Dyrcona joined #evergreen |
| 12:23 |
miker |
Dyrcona: https://pastebin.com/csTYASPu is what I'm proposing. commit coming in a moment |
| 12:45 |
miker |
Dyrcona: 2 new commits on the QI branch now, though just the first is strictly required for the browse ingest conflict. the second is about a long-standing oversight that removes too much data in a "just update this one field" reingest request |
| 12:47 |
miker |
(IOW, you may want to pick and test just the first new commit, but I think the second is needed generally) |
| 12:49 |
|
collum joined #evergreen |
| 12:55 |
|
kworstell-isl joined #evergreen |
| 12:59 |
Dyrcona |
miker: I'll try them both in a bit. |
