| Time |
Nick |
Message |
| 06:01 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 06:39 |
|
alynn26_away joined #evergreen |
| 07:31 |
|
rjackson_isl_hom joined #evergreen |
| 08:26 |
|
rfrasur joined #evergreen |
| 08:53 |
|
Dyrcona joined #evergreen |
| 08:54 |
|
mantis joined #evergreen |
| 08:58 |
|
mmorgan joined #evergreen |
| 09:07 |
Dyrcona |
So, running an upgrade from 3.5.3 to 3.7.2 and the record reingest from 1252 is taking a long time. The whole db upgrade has been running for 22 hours and 57 minutes. I think I can speed it up by just having the search format attribute ingested. |
| 09:08 |
Dyrcona |
Aside: I find it amusing that the psql process has id 11111. |
| 09:21 |
Dyrcona |
I think this will be faster https://pastebin.com/k9YChekC |
| 09:23 |
Dyrcona |
Yeah, 6 minutes versus over 20 hours. |
| 09:24 |
Dyrcona |
1258 looks like it might want a reingest, too. |
| 09:40 |
Dyrcona |
Here's a reingest for the 1258 upgrade if you want it: https://pastebin.com/vyjPjYpG |
| 09:41 |
Dyrcona |
Maybe I should Lp those? |
| 09:41 |
alynn26 |
+1 for that |
| 09:56 |
Dyrcona |
Lp 1955154 |
| 09:56 |
pinesol |
Launchpad bug 1955154 in Evergreen "Search Fromat Reingest from 1252 DB Upgrade Is Slow" [Undecided,New] https://launchpad.net/bugs/1955154 |
| 10:01 |
Dyrcona |
Lp 1955156 |
| 10:01 |
pinesol |
Launchpad bug 1955156 in Evergreen "Reignest SQL Missing for 1258 DB Upgrade" [Undecided,New] https://launchpad.net/bugs/1955156 |
| 10:01 |
mmorgan |
Dyrcona++ |
| 10:02 |
Dyrcona |
That latter ingest is slow, but there's not much else one can do with it. It's based on number of bibs in your database. |
| 10:12 |
* Dyrcona |
sighs. |
| 10:13 |
Dyrcona |
1270 is also missing specific ingest code. I suppose one is expected to do a full reingest after every upgrade, but with Did You Mean breaking parallel ingest, I'm inclined not to. |
| 10:25 |
Dyrcona |
Lp 1955158 |
| 10:25 |
pinesol |
Launchpad bug 1955158 in Evergreen "Reingest SQL Missing for the 1270 DB Upgrade" [Undecided,New] https://launchpad.net/bugs/1955158 |
| 11:32 |
|
jvwoolf joined #evergreen |
| 12:09 |
|
jihpringle joined #evergreen |
| 12:23 |
csharp_ |
Dyrcona++ |
| 12:24 |
csharp_ |
we're going to do the full reingest with the retain_sympell triggers disabled |
| 12:26 |
Dyrcona |
csharp_: I was just thinking about symspell and reingest. I haven't looked at the code, yet, but I was thinking it could be made into its own ingest process, so it could be run separate from the others in pingest.pl. It would then work like the browse search does with the whole thing done in one process. |
| 12:31 |
Dyrcona |
Triggers may have to go. |
| 12:31 |
Dyrcona |
We'll see.... |
| 12:32 |
Dyrcona |
csharp_: How are you doing the full ingest? I recommend against messing wit flags and update biblio record entry. |
| 12:41 |
Dyrcona |
My plan when upgrading to 3.7 later is to just run the ingests that are needed and then do the symspell set up. |
| 13:03 |
rjackson_isl_hom |
always try to keep disruption of the community at a minimum - but we are having issues trying to rerun our autorenewals today |
| 13:03 |
rjackson_isl_hom |
rest all triggered events to pending and restarted the script for the granularity in question (Daily) |
| 13:03 |
rjackson_isl_hom |
log doesn't show it firing up |
| 13:04 |
rjackson_isl_hom |
did similar for Daily 3 day and found this: 2021-12-17 12:35:21 utility-prod open-ils.trigger: [INFO:2391:Application.pm:159:] CALL: open-ils.trigger open-ils.trigger.passive.event.autocreate.batch expire, home_ou, HASH(0x6191370), Daily-3dayOver |
| 13:05 |
rjackson_isl_hom |
so my question is does the processing create some type of lock in a table and we need to wipe it out to get a clean restart? |
| 13:15 |
Dyrcona |
rjackson_isl_hom: No, it doesn't. Use the --run-pending switch with your action trigger command. |
| 13:16 |
rjackson_isl_hom |
Dyrcona I checked to ensure that it there (--run-pending) |
| 13:18 |
Dyrcona |
Your log message looks perfectly normal for an event creation. |
| 13:19 |
rjackson_isl_hom |
the log entry is for a different granularity though |
| 13:19 |
rjackson_isl_hom |
I have two attemtps running to try and prcoess things for consortium that initially died in the early morning hours |
| 13:20 |
rjackson_isl_hom |
one for Daily and one for Daily-sdayOver |
| 13:20 |
rjackson_isl_hom |
Daily-3dayOver |
| 13:21 |
rjackson_isl_hom |
the daily should be going after the Autorenewals |
| 13:21 |
rjackson_isl_hom |
and I can imagine the heat if a few thousand don't happen today1 |
| 13:22 |
Dyrcona |
rjackson_isl_hom: When I have stuck autorenewals, I run this: https://pastebin.com/vjAAYcwt |
| 13:23 |
Dyrcona |
Then I run this on the utility server: action_trigger_runner.pl --osrf-config /openils/conf/opensrf_core.xml --run-pending --granularity daily --granularity-only |
| 13:24 |
Dyrcona |
Note that in the SQL I pasted there's a placeholder for the autorenewal event id. Ours might be different from yours. |
| 13:25 |
Dyrcona |
If it keeps blowing up, you probably have a patron with "too many" items checked out to renew, where "too many" can vary. In that case, I recommend also increasing the max stanza size in ejabberd to 10 or 20 million. |
| 13:28 |
csharp_ |
Dyrcona: I was going to run this script: https://git.evergreen-ils.org/?p=evergreen/pines.git;a=blob;f=Open-ILS/src/sql/Pg/version-upgrade/run_pines_upgrade.sh;h=3c4d8d6415ec35a49a8c5fc4a1137ec4c3d6a7ac;hb=refs/heads/rel_3_8_0 - see the last section |
| 13:30 |
Dyrcona |
csharp_: I'd be interested to know how that works out. |
| 13:38 |
rjackson_isl_hom |
Dyrcona++ probably a staff account with 150 renewals today :( per alynn26 |
| 13:56 |
Dyrcona |
Yeah, those accounts bother me. They're almost always used for something that a status, copy location, or even a bucket would server better. |
| 13:57 |
alynn26 |
I am with you Dyrcona |
| 13:57 |
Dyrcona |
Though, I have seen a children's librarian checkout 30+ items for story hour or whatever....Not sure why the feel compelled to check things out all the time. |
| 13:58 |
Dyrcona |
Things like "Missing" cards have no business existing as far as I'm concerned. |
| 14:25 |
mmorgan |
That Mr. Missing always has soooo many overdues! |
| 14:26 |
alynn26 |
When i find a mr missing, due dates get set to the far future. |
| 14:27 |
mmorgan |
Any way that type of card can be filtered out of the autorenewals? |
| 14:27 |
Dyrcona |
mmorgan: Probably, yes, if it has a particular profile. |
| 14:28 |
mmorgan |
Might avoid some issues. |
| 14:28 |
jeffdavis |
When I see that kind of staff account, I wonder if alternate workflows are more cumbersome than they should be. |
| 14:28 |
Dyrcona |
One reason for these cards I was given at MVLC is they wanted the items to go to lost so that they could calculate the value of the missing items from the lost fees on the account. |
| 14:29 |
alynn26 |
jeffdavis, Or, it was the way it worked before why learn something new. |
| 14:29 |
jeffdavis |
That too, but I'm trying to be less cynical :P |
| 14:30 |
Dyrcona |
I'm inclined to think it's a case of "That's how we've always done it." Y'know since it was the only way to do it 3 systems ago. |
| 14:30 |
* mmorgan |
would say the Mark Missing workflow is definitely less cumbersome than retrieving the Missing patron and checking out the item. |
| 14:32 |
jeffdavis |
One of our libraries has a very complicated workflow for transferring entire collections between branches every 3-6 months that was causing system-wide problems due to "parallel requests" bugs. That was definitely a mix of "other methods are too cumbersome" and "this is how we've always done it." (Now we're doing it for them in the database.) |
| 14:48 |
|
rjackson_isl_hom joined #evergreen |
| 15:00 |
Dyrcona |
Does changing an org unit name require running autogen.sh? |
| 15:03 |
Dyrcona |
Yeah, looks like it would be a good idea. |
| 15:03 |
mmorgan |
Dyrcona: I have some notes that indicate the answer is "Yes" |
| 15:08 |
Dyrcona |
Heh. mmorgan++ |
| 15:08 |
Dyrcona |
It's also the kind of thing that I should just know after how many years....? |
| 15:16 |
* mmorgan |
should also know right from left after how many years, but... |
| 15:24 |
Dyrcona |
:) |
| 15:47 |
Bmagic |
Dyrcona: name and shortname are things that need to change at night on production so that autogen.sh can be ran and staff sessions don't get confused when doing pcrud stuff. And the other columns in actor.org_unit I'm less certain about |
| 15:47 |
Bmagic |
opac_visible too :) |
| 16:22 |
|
jvwoolf left #evergreen |
| 17:01 |
* miker |
reads up re ingest and mumbles something about queued ingest and a 7.5-ish year old plan and lacking time and tuits, and wanders off |
| 17:02 |
* mmorgan |
wishes many holiday tuits to all! |
| 17:04 |
miker |
@later tell Dyrcona it may be worth confirming that the record attribute vector vlist column for the records you did a partial attirbute reingest on didn't get blown away and replaced with /only/ the attribute you specified. IIRC, I've seen that happen. if it did, it's obv a bug to be fixed for the good of speedier ingest. (also, IIUC, the symspell fixes post-3.7.0 should have addressed the deadlocks -- or at least made them excedingly rare) |
| 17:04 |
pinesol |
miker: The operation succeeded. |
| 17:06 |
|
mmorgan left #evergreen |
| 17:19 |
|
rjackson_isl_hom joined #evergreen |
| 18:01 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 19:47 |
|
jihpringle joined #evergreen |
