| Time |
Nick |
Message |
| 00:52 |
|
troy joined #evergreen |
| 06:01 |
pinesol |
News from qatests: Failed Installing Evergreen database pre-requisites <http://testing.evergreen-ils.org/~live//archive/2022-02/2022-02-14_04:00:02/test.39.html> |
| 07:44 |
|
collum joined #evergreen |
| 08:17 |
|
Dyrcona joined #evergreen |
| 08:37 |
Dyrcona |
Another stuck search query running for 18 hours, well, two, but they look like the same search in the database. I guess the patron/staff tried it again when it didn't return any results. |
| 08:38 |
Dyrcona |
Also, why are ISBN and date split functions being run on the same input. That just seems wrong, and hand waving about "it can happen if..." isn't good enough. |
| 08:47 |
|
mmorgan joined #evergreen |
| 08:55 |
|
terranm joined #evergreen |
| 11:12 |
|
jvwoolf joined #evergreen |
| 11:13 |
Dyrcona |
We had some libraries leave a few years ago. While looking into adding records for an e-resource vendor, I stumbled across asset.uri entries from this vendor for one of the libraries that left. |
| 11:15 |
Dyrcona |
I though we must have some 856s still in the database, but no. There are no links to asset.uri_call_number_map for these asset.uri entries, no asset.call_number entries, and I didn't find any metabib.real_full_rec entries when I did a tsquery on index_vector. |
| 11:15 |
Dyrcona |
Anyone else seen orphaned asset.uri entries before? |
| 11:16 |
Dyrcona |
I was able to just delete them from a test database. |
| 11:16 |
mmorgan |
bug 1482757 rears its ugly head :) |
| 11:16 |
pinesol |
Launchpad bug 1482757 in Evergreen "Loading records with located URIs should not delete and recreate call_numbers" [Low,Confirmed] https://launchpad.net/bugs/1482757 |
| 11:18 |
Dyrcona |
Yeah, maybe, but this feels like a different bug. I'll have to reread that one to see if orphaned entries are mentioned. |
| 11:22 |
mmorgan |
Dyrcona: it does: https://bugs.launchpad.net/evergreen/+bug/1482757/comments/5 |
| 11:22 |
pinesol |
Launchpad bug 1482757 in Evergreen "Loading records with located URIs should not delete and recreate call_numbers" [Low,Confirmed] |
| 11:22 |
Dyrcona |
Intereestingly, I found 20,000+ entries in metabib.real_full_rec that reference this particular libary in the 856, but no subfield u, just 9, y and z. |
| 11:22 |
Dyrcona |
GIGO. |
| 11:23 |
Dyrcona |
mmorgan: Cool, I'll read the bug description again. IIRC, I already signed off on the branch, no? I don't think it made much difference in performance when I tested it. |
| 11:24 |
* mmorgan |
needs to refresh memory on that, too. We are running some version of that patch in production. |
| 11:25 |
mmorgan |
Not sure it makes much difference in performance, but does eliminate call number churn. |
| 11:34 |
Dyrcona |
Right. I think I'm just going to do a blanket search for orphaned asset.uri entries and delete them. |
| 11:36 |
Dyrcona |
I also found 10,042 not deleted bibs with 856 tags that reference this particular library. That's a total separate (local) issue, but I based the delete on there being asset.call_number entries for the libraries that were leaving. These might not have had those, or I might have had a bug in my query. |
| 11:37 |
* jvwoolf |
wonders if this bug could contribute to the issue we've seen with reports filtering on call number timing out |
| 11:39 |
* Dyrcona |
tries to remember the trick for deleting with a left join...Thinks it was probably a "with" query. |
| 11:39 |
Dyrcona |
jvwoolf: Could be. |
| 11:42 |
jvwoolf |
Our asset.call_number table doesn't seem super huge in comparison to our asset.copy table |
| 11:48 |
Dyrcona |
Hmm. 1,469,928 asset.uri entries with no link to asset.uri_call_number_map.... |
| 11:48 |
Dyrcona |
That's over half the rows in the table. |
| 11:53 |
mmorgan |
jvwoolf: Do you have a lot of records with 856 links? How big is your asset.uri table? |
| 11:54 |
jvwoolf |
mmorgan: Not very big. 727,037 rows. |
| 11:55 |
jvwoolf |
asset.uri_call_number_map is fairly big though |
| 11:55 |
jvwoolf |
5,429,720 rows in that table |
| 12:04 |
mmorgan |
Our asset.uri is 2,748,091. asset.uri_call_number_map is 2,749,956. Again, we're running a version of patch from 1482757 |
| 12:12 |
|
jihpringle joined #evergreen |
| 12:21 |
jvwoolf |
Well, 'select count(id) from asset.call_number where label = '##URI##'' and deleted = TRUE;' Yields 30,942,609 rows |
| 12:21 |
jvwoolf |
So I must have read that total number of rows wrong... |
| 12:22 |
Dyrcona |
Well, 30 million deleted rows sounds about right... ;) |
| 12:23 |
jvwoolf |
The whole table has 44,243,364 |
| 12:23 |
jvwoolf |
So that's what, like 2/3 of the table? Yikes! |
| 12:30 |
mmorgan |
We have 18,509,338 deleted URI call numbers, 2,714,979 UNdeleted URI call numbers, 28,346,758 total rows in asset.call_number |
| 12:35 |
Dyrcona |
Over 2/3 of our asset.call_number table is deleted = 't' rows. I didn't bother to count those with the ##URI## label. |
| 12:38 |
|
jihpringle joined #evergreen |
| 12:44 |
|
jihpringle68 joined #evergreen |
| 12:52 |
Dyrcona |
Maybe it's high time that patch went in. I signed off over a year ago, but that probably means it needs a rebase and to be tested again. |
| 12:58 |
jvwoolf |
Dyrcona: If you rebase it, I can test it on a 9.6 DB with production data |
| 12:58 |
jvwoolf |
We're only running Evergreen 3.6 though |
| 12:59 |
jvwoolf |
Also, I see that there's an upgrade query in here to get rid of the orphaned entries in asset.uri_call_number_map but nothing to get rid of the call numbers |
| 13:00 |
jvwoolf |
I think that probably needs to be addressed in the release notes at the very least. It's probably better handled outside the upgrade scripts since it would be deleting the majority of the asset.call_number table in our case. |
| 13:03 |
Dyrcona |
jvwoolf: You can't really get rid of the call number. They just get tagged deleted. Or, do you mean, that there should be something to set call numbers deleted, too? |
| 13:04 |
jvwoolf |
Dyrcona: I think for us, we're going to need to get rid of the call number or our reports for filtering on call number are going to stay broken |
| 13:04 |
jvwoolf |
So maybe we're a special case |
| 13:10 |
Dyrcona |
IDK, we have 18+ million deleted call numbers, not quite as many as you. Unfortunately, you can't just delete them without jumping through some serious hoops, and you'll probably want to block access to the database while doing so. |
| 13:13 |
mmorgan |
Yeah, actually deleting the rows would be hard. Because we're running a version of that patch, we're not generating as many newly deleted call numbers as we would have been, but we haven't addressed the already deleted rows. |
| 13:18 |
Dyrcona |
jvwoolf: BTW, I sent bshum a SQL to remove unused bib records years ago when he still worked at Bibliomation. You guys hired me to write it. With some tweaking, it could be updated to remove just asseet.call_number entries. |
| 13:46 |
jvwoolf |
Dyrcona: Thank you I'll take a look around to see if that is saved somewhere |
| 14:17 |
|
jihpringle joined #evergreen |
| 16:07 |
|
jihpringle joined #evergreen |
| 16:30 |
|
Keith-isl joined #evergreen |
| 16:51 |
|
jvwoolf left #evergreen |
| 17:10 |
|
mmorgan left #evergreen |
| 18:01 |
pinesol |
News from qatests: Failed Installing Evergreen database pre-requisites <http://testing.evergreen-ils.org/~live//archive/2022-02/2022-02-14_16:00:02/test.39.html> |
