| Time |
Nick |
Message |
| 06:35 |
|
TARA joined #evergreen |
| 06:57 |
|
lualaba joined #evergreen |
| 06:58 |
lualaba |
Hello. how i can regenerate asset.uri_call_number_map and asset.uri from marc records? |
| 07:02 |
|
lualaba joined #evergreen |
| 07:02 |
lualaba |
i know there is function biblio.extract_located_uris, but how to run for all records |
| 07:11 |
|
JBoyer joined #evergreen |
| 07:16 |
csharp |
lualaba: you might try 'select biblio.extract_located_uris(id, marc, <putyouruseridherewithoutbrackets>) from biblio.record_entry where not deleted;' |
| 07:16 |
csharp |
lualaba: that's totally untested and I'm not sure what your ultimate goal is, so use with caution :-) |
| 07:17 |
csharp |
lualaba: also note that if your biblio.record_entry table is large, that may take a long time and use a lot of I/O |
| 07:19 |
lualaba |
Thank you i have 30K records, after migration something is wrong with URI and i need regenerate |
| 07:21 |
lualaba |
if you know another way to fix this problem please tell me |
| 07:22 |
csharp |
sorry - I'm not familiar enough to recommend anything - someone else may be able to help - it's still early in the Eastern time zone - you might ask again in a couple of hours |
| 07:23 |
csharp |
@who is an early riser? |
| 07:23 |
pinesol_green |
gmcharlt is an early riser. |
| 07:24 |
lualaba |
ok i will ask after a couple of hours |
| 07:33 |
|
rjackson_isl joined #evergreen |
| 07:39 |
rhamby |
morning |
| 07:39 |
jeff |
morning. |
| 07:52 |
|
TARA joined #evergreen |
| 08:30 |
|
collum joined #evergreen |
| 08:32 |
|
Dyrcona joined #evergreen |
| 08:40 |
|
mmorgan joined #evergreen |
| 08:44 |
|
ericar joined #evergreen |
| 08:49 |
|
wjr joined #evergreen |
| 08:50 |
|
wjr joined #evergreen |
| 09:00 |
|
Newziky joined #evergreen |
| 09:04 |
|
mrpeters joined #evergreen |
| 09:05 |
|
_bott_ joined #evergreen |
| 09:10 |
kmlussier |
@coin |
| 09:10 |
pinesol_green |
kmlussier: heads |
| 09:18 |
|
mrpeters1 joined #evergreen |
| 09:20 |
|
mmorgan1 joined #evergreen |
| 09:28 |
csharp |
@praise [coin] |
| 09:28 |
* pinesol_green |
Shall I compare heads to a summer's day? heads is more lovely and more temperate. |
| 09:31 |
Dyrcona |
@coin |
| 09:31 |
pinesol_green |
Dyrcona: tails |
| 09:50 |
|
yboston joined #evergreen |
| 10:10 |
|
jwoodard joined #evergreen |
| 11:17 |
|
vlewis joined #evergreen |
| 11:24 |
|
Christineb joined #evergreen |
| 11:40 |
|
Callender joined #evergreen |
| 11:43 |
|
edoceo joined #evergreen |
| 12:09 |
|
sandbergja joined #evergreen |
| 12:22 |
|
brahmina joined #evergreen |
| 12:25 |
|
bmills joined #evergreen |
| 13:07 |
|
jihpringle joined #evergreen |
| 13:10 |
|
mmorgan joined #evergreen |
| 13:15 |
|
mmorgan joined #evergreen |
| 13:24 |
Dyrcona |
Someone from BC around who can answer some questions about Novelist On the Shelf? |
| 13:25 |
Dyrcona |
I'd like to know if all of your member have it, or just Whistler. |
| 13:25 |
Dyrcona |
Also, did you have to do anything special in the OPAC, or does it work with the existing Novelist code? |
| 13:30 |
jeffdavis |
Not everyone in Sitka has it, but more than just Whistler I think ... let me check. |
| 13:33 |
jeffdavis |
Dyrcona: yeah, we have maybe half a dozen sites using it |
| 13:33 |
Dyrcona |
Thanks, did you have to install any special code for them? |
| 13:34 |
jeffdavis |
doesn't look like it, all we seem to do is suppress "Awards, Reviews, & Suggested Reads" for non-Novelist sites |
| 13:34 |
Dyrcona |
When you send the extract, is it all the bibs in the consortium? |
| 13:34 |
* Dyrcona |
has lots of questions.... |
| 13:35 |
jeffdavis |
sorry, not sure what you mean? |
| 13:35 |
gmcharlt |
dbs: calling your attention to bug 1563531 with a potential Fedora dependency question |
| 13:35 |
pinesol_green |
Launchpad bug 1563531 in Evergreen "Missing libpcre during installation ./config EG 2.10.1" [Medium,Triaged] https://launchpad.net/bugs/1563531 |
| 13:36 |
Dyrcona |
You have to send them a monthly record extract. I wanted to know if it included records for the whole consortium or only those libraries subscribed to on the shelf. |
| 13:36 |
Dyrcona |
One of our libraries is asking us about getting on the shelf. |
| 13:37 |
jeffdavis |
Sounds like the libraries that are using Novelist use the reporter to generate whatever they need to send off. |
| 13:37 |
Dyrcona |
jeffdavis: Do you have libraries using Novelist that do not use on the shelf? |
| 13:37 |
Dyrcona |
OK. |
| 13:37 |
Dyrcona |
They have instructions on how to do that on the novelist site, but I was going to set something up myself. |
| 13:38 |
jeffdavis |
Yeah, as you can probably tell we leave it to the individual libraries to manage. It's not something I've looked at. |
| 13:39 |
Dyrcona |
jeffdavis++ # Your answers are helpful. |
| 13:39 |
jeffdavis |
oh good! |
| 13:44 |
dbs |
thanks gmcharlt |
| 13:45 |
kmlussier |
yboston: Less than half a mile from the conference hotel - http://woodyscitymarket.com/events/action~month/exact_date~1459532411/request_format~html/ |
| 13:46 |
csharp |
kmlussier++ |
| 13:55 |
JBoyer |
Dyrcona: We have OTS system wide and automate the data it needs. Let me look at what we're doing. (I can't answer the "What about libs not using OTS" part though, since all of ours are.) |
| 13:55 |
Dyrcona |
Thanks, JBoyer. I think I've got it sorted. |
| 13:56 |
Dyrcona |
This one library has its own code with Novelist/EBSCO, so there should be no problems with OTS showing up. |
| 13:56 |
Dyrcona |
I can tell them to use the extract for that profile. |
| 13:56 |
JBoyer |
Good to hear! That does keep it simpler. |
| 13:59 |
Dyrcona |
What I'll have to find out is do I send the extract for all of the consortium's items or just this one library's. |
| 14:01 |
JBoyer |
If they've got their own Novelist code I'd think only theirs, but yeah, I'd verify it. |
| 14:05 |
|
gsams joined #evergreen |
| 14:06 |
yboston |
kmlussier++ |
| 14:21 |
|
mmorgan joined #evergreen |
| 14:26 |
Dyrcona |
JBoyer: You have a script that you run for the extract, or do you do it from the reports? |
| 14:27 |
JBoyer |
A script. Fie on reports. |
| 14:27 |
Dyrcona |
Could you share it? Might save me a few hours of work. |
| 14:35 |
pastebot |
"JBoyer" at 64.57.241.14 pasted "rjackson_isl put this together for us based somewhat on the CollectionHQ extract scripts" (57 lines) at http://paste.evergreen-ils.org/11 |
| 14:37 |
|
tarac_ joined #evergreen |
| 14:37 |
Dyrcona |
JBoyer++ # Thanks! |
| 14:49 |
|
lualaba joined #evergreen |
| 14:49 |
lualaba |
hello. what is best way to regenerate asset.uri and asset.uri_call_number_map. |
| 14:50 |
lualaba |
when i make simple marc update after that i able see uris in opac |
| 14:51 |
lualaba |
i make migration ffrom 2.8 to 2.10 |
| 14:51 |
Dyrcona |
lualaba: If you change the 856 in an appropriate way, they should update automatically. |
| 14:52 |
lualaba |
i don't tpuch marc fields. |
| 14:52 |
lualaba |
and how to change for all records automatically? |
| 14:54 |
|
lualaba_ joined #evergreen |
| 14:59 |
dbs |
lualaba_: I think you would be looking at something like setting ingest.reingest.force_on_same_marc to TRUE in config.internal_flag and updating each record with something like "UPDATE biblio.record_entry SET id = id;" (or subsets thereof for large databases) |
| 15:00 |
jeff |
and if you suspect issues due to bad migrated data, you might want to run it on a very small number of records and examine the results first. |
| 15:01 |
lualaba_ |
It is very small number of records appr 2 K |
| 15:03 |
|
ericar joined #evergreen |
| 15:15 |
lualaba_ |
i will check thank you |
| 15:17 |
jeffdavis |
psql:2.9.3-2.10.0-upgrade-db.sql:4580: ERROR: insert or update on table "usr_circ_history" violates foreign key constraint "usr_circ_history_target_copy_fkey" |
| 15:17 |
jeffdavis |
DETAIL: Key (target_copy)=(9680394) is not present in table "copy". |
| 15:17 |
jeffdavis |
O_o |
| 15:18 |
jeffdavis |
^ Anyone else run into that during a 2.10 upgrade? |
| 15:19 |
berick |
jeffdavis: hmm |
| 15:19 |
berick |
so, i guess target_copy is not a foreign key ref in action.aged_circulation |
| 15:20 |
berick |
iow, you can have an aged circ that points at a copy that does not exist |
| 15:20 |
jeff |
I can confirm that I have 54 action.circulation rows that reference a target_copy that no longer exists in asset.copy. |
| 15:21 |
mrpeters1 |
afternoon guys -- does USMARC = MARC21 Binary with regard to the marc_export tool? |
| 15:21 |
berick |
jeff: whoa, it's not even a ref on action.circulation. |
| 15:21 |
berick |
arg |
| 15:22 |
jeff |
nope, no target_copy constraints (foreign or otherwise) on action.circulation or action.aged_circulation in our 2.7ish db. |
| 15:22 |
berick |
ok, so, we'll have to modify the upgrade script to do .. what .. with these circs? a checkout history item w/ no copy makes no sense, so I guess it should skip them? |
| 15:22 |
berick |
the aged circs will still be there |
| 15:24 |
berick |
jeffdavis: i'll open a bug. |
| 15:24 |
gmcharlt |
mrpeters1: yep |
| 15:25 |
jeffdavis |
Oddly, I think that's actually the issue we're running into. We don't seem to have any target_copy values without corresponding asset.copy.id's. |
| 15:25 |
jeffdavis |
s/actually/actually not/ |
| 15:25 |
jeff |
berick: i think skipping them is probably best. jumping through hoops to permit "checkout of unknown item 123445" is probably not worth it. |
| 15:25 |
mrpeters1 |
gmcharlt: thanks much appreciated! OCLC requests MARC21 with MARC8 encoding -- just wanted to verify the -f switch was right |
| 15:26 |
berick |
jeffdavis: in action.circulation? |
| 15:26 |
berick |
jeff: *nod* |
| 15:26 |
jeff |
jeffdavis: zero results for the following? SELECT COUNT(*) FROM action.circulation circ LEFT JOIN asset.copy acp ON circ.target_copy = acp.id WHERE acp IS NULL; |
| 15:27 |
jeff |
jeffdavis: and same but s/circulation/aged_circulation/ ? |
| 15:27 |
jeffdavis |
correct, for both action.circulation and action.aged_circulation |
| 15:28 |
mmorgan |
jeffdavis: jeff: I'm confused. Do you actually delete rows from asset.copy? |
| 15:28 |
jeff |
mmorgan: our database schema and data have a long and storied history. |
| 15:28 |
jeff |
mmorgan: and should not be taken as typical of anyone else's database. :-) |
| 15:28 |
mmorgan |
gotcha. |
| 15:29 |
berick |
jeffdavis: you have an asset.copy with id = 9680394 ? |
| 15:29 |
jeffdavis |
berick: yes |
| 15:29 |
jeffdavis |
I need to confirm if it gets deleted somehow during the upgrade transaction |
| 15:31 |
|
mmorgan1 joined #evergreen |
| 15:31 |
berick |
jeffdavis: if you have such a copy and it's not deleted in the upgrade xact, then I'm confused by the error. |
| 15:31 |
Dyrcona |
That's strange. I've run that script two or three times now and not had a problem with it. |
| 15:35 |
jeffdavis |
Yeah, the asset.copy record with id = 9680394 still exists just before COMMIT. |
| 15:36 |
jeffdavis |
Maybe this db snapshot is corrupt or something. |
| 15:37 |
jeff |
test by creating a dummy table with a BIGINT NOT NULL REFERENCES asset.copy(id) and then try to insert 9680394 to that field? |
| 15:37 |
Dyrcona |
Doesn't look like that part of the upgrade scripts does any deletes. |
| 15:37 |
berick |
jeff: i'm guessing you have not tried the 2.10 upgrade script yet... |
| 15:37 |
jeff |
it's possible that a bad index could be causing you grief... |
| 15:38 |
jeff |
berick: i can confirm that, but expect to this week or next. :-) |
| 15:38 |
* Dyrcona |
is looking at 0960 fwiw. ;) |
| 15:40 |
jeffdavis |
jeff: good idea! Yup, creating that table and inserting that value gives the same error. |
| 15:40 |
Dyrcona |
jeff++ |
| 15:41 |
|
vlewis_ joined #evergreen |
| 15:42 |
jeff |
so, perhaps the next step would be REINDEX INDEX asset.copy_pkey; |
| 15:42 |
jeff |
and possibly check for any duplicate index on that column. |
| 15:44 |
|
vlewis joined #evergreen |
| 15:44 |
jeffdavis |
REINDEX on that index and on the table don't help, nor did VACUUM ANALYZE. I'll try VACUUM FULL. |
| 15:45 |
|
vlewis joined #evergreen |
| 15:46 |
pastebot |
"berick" at 64.57.241.14 pasted "jeff -- does this return >0 for you?" (9 lines) at http://paste.evergreen-ils.org/13 |
| 15:47 |
jeff |
jeffdavis: what version of postgres? |
| 15:48 |
jeff |
berick: 4 :-) |
| 15:49 |
jeff |
berick: but please see previous statement regarding our db having a long and storied history. :-) |
| 15:49 |
berick |
jeff: ugh, well, i was just wondering what the odds of this actually being a problem are |
| 15:49 |
berick |
but, i guess 100% in your case :) |
| 15:50 |
* berick |
will continue opening bug |
| 15:50 |
* tsbere |
wonders if there is a difference between "select from asset.copy where id = blah" and "select from only asset.copy where id = blah" in this case |
| 15:51 |
* tsbere |
has also not actually been following anything |
| 15:52 |
tsbere |
jeffdavis / jeff/ berick : Do you all remember that serial.unit inherits from asset.copy so all serial.unit entries are also asset.copy entries, but that as a result foreign keys don't work to asset.copy.id? |
| 15:53 |
* tsbere |
bases this question on the comment about testing with a REFERENCES asset.copy(id) earlier |
| 15:54 |
jeffdavis |
tsbere: ah! |
| 15:54 |
berick |
trying to remember how serials + circulation works now.. |
| 15:54 |
Dyrcona |
jeffdavis: That would be something to check if the id exists in serial.unit. |
| 15:54 |
JBoyer |
While the subject of missing constraints has come up, there's also not one between ahr.cancel_cause and ahrcc.id. It slipped my mind because it wasn't the cause of the problem I was trying to figure out at the time. Since it's a one-liner it could be lumped in to a larger "constrain all the things!" bug.. |
| 15:55 |
tsbere |
jeffdavis: That is why I mentioned "from asset.copy" vs "from only asset.copy" - the latter won't show serial.unit entries |
| 15:55 |
Dyrcona |
Well, it sounds like constraints missing from action.circulation may be on purpose. |
| 15:55 |
jeffdavis |
Yeah, id 9680394 exists in serial.unit |
| 15:55 |
Dyrcona |
tsbere++ |
| 15:56 |
* tsbere |
is now curious what he has actually helped with because he came in late and has no clue, and doesn't feel like trying to find the right starting point in the scrollback |
| 15:57 |
berick |
i think it boils down to: can you check out serial.unit's (via action.circulation) ? |
| 15:57 |
* berick |
doesn't remember |
| 15:57 |
Dyrcona |
berick: I think you can. |
| 15:57 |
tsbere |
berick: Yes, you can |
| 15:57 |
Dyrcona |
We don't use serials, though. |
| 15:57 |
tsbere |
berick: From the point of view of everything but foreign key checks serial.unit entries act like asset.copy entries as well |
| 15:57 |
Dyrcona |
Well, let me say, AFAIK, we don't use serials. :) |
| 15:57 |
berick |
OK, then that's the real issue. |
| 15:58 |
berick |
checkout history does not take serials into account |
| 15:58 |
tsbere |
berick: A foreign key will only check the table, not inheriting tables. Thus, select * from only asset.copy shows you what a FK will see, while select * from asset.copy shows you everything. |
| 15:58 |
Dyrcona |
Also explains whey we've not had that problem since we don't use serials. |
| 15:58 |
jeffdavis |
So we can just modify action.usr_circ_history so the target_copy table references serial.unit instead? |
| 15:58 |
tsbere |
jeffdavis: No, it needs to not be a FK at all |
| 15:58 |
berick |
jeff: also curious if this is what's happening in your case... ^-- |
| 15:59 |
tsbere |
jeffdavis: Every serial.unit entry shows up in asset.copy (without the only keyword) - The opposite is not true. |
| 16:01 |
|
brahmina joined #evergreen |
| 16:04 |
jeff |
berick: no, i still have 4 output rows for your query and 54 circs overall whose target_copy does not exist in asset.copy or serial.unit. |
| 16:05 |
berick |
jeff: k, thanks for checking |
| 16:05 |
berick |
will address both in the LP |
| 16:06 |
jeff |
we have ~86k circs referencing serial.unit rows, so we'd hit the FK issue as well. |
| 16:06 |
jeff |
good thought tsbere++ |
| 16:07 |
jeff |
well, we'd hit the FK issue for other reasons as well, but also for serial.unit reasons. :-) |
| 16:08 |
berick |
yeah, tsbere++ |
| 16:10 |
jeffdavis |
tsbere++ ; jeff++ ; berick++ # thanks all! |
| 16:12 |
Dyrcona |
bshum mentioned lp 1152753 to me in relation to this discussion. |
| 16:12 |
pinesol_green |
Launchpad bug 1152753 in Evergreen "Serial Units won't go into Copy Buckets" [Medium,Confirmed] https://launchpad.net/bugs/1152753 |
| 16:12 |
* Dyrcona |
starts to think that maybe table inheritance is not always a good thing. ;) |
| 16:15 |
jeffdavis |
Well, removing the FK reference on action.usr_circ_history.target_copy did allow the upgrade script to commit without errors. |
| 16:15 |
jeff |
> These deficiencies will probably be fixed in some future release, but in the meantime considerable care is needed in deciding whether inheritance is useful for your application. |
| 16:16 |
Dyrcona |
:) |
| 16:16 |
berick |
jeffdavis: thanks for testing and confirming that |
| 16:17 |
jeff |
that statement (or a singular variant of it) has been in the Postgresql docs at least as early as 7.3 :-) |
| 16:17 |
Dyrcona |
Well, jeff, you can always step up to write the code to address the deficiencies. :) |
| 16:17 |
jeffdavis |
berick: easily done, and now I have a 2.10 server to play with - thanks for the help! |
| 16:18 |
* tsbere |
knows the foreign key issue also shows up in transits |
| 16:18 |
tsbere |
Can't FK to a transit because of hold transits |
| 16:18 |
* Dyrcona |
tries to figure out where he should create a new database function: i.e. which schema and which git repo. |
| 16:18 |
jeff |
(And I'm not arguing here that we should stop using inheritance -- just amused at the "forward-looking statements" in the postgresql docs. :-) ) |
| 16:19 |
Dyrcona |
# TODO: Make it actually work. :) |
| 16:19 |
|
jlitrell joined #evergreen |
| 16:20 |
tsbere |
jeff: Can *I* argue that we stop using inheritance for serials and just make the extra fields available on a secondary table with a FK to asset.copy? ;) |
| 16:20 |
* tsbere |
thinks that would save a lot of headaches when migrating people *into* using serials as well, you could re-use the existing copies... |
| 16:20 |
jeff |
tsbere: patches welcome! |
| 16:20 |
Dyrcona |
heh |
| 16:20 |
jeff |
:-) |
| 16:20 |
* tsbere |
doesn't know enough about using serials to test anything he did write, so will have to pass on writing the changes himself <_< |
| 16:21 |
* Dyrcona |
thinks someone who uses serials should do that one. :) |
| 16:30 |
* miker |
scrolls up... |
| 16:38 |
|
abneiman joined #evergreen |
| 16:41 |
* jeffdavis |
wonders if miker is still stranded somewhere in all that scrollback |
| 16:43 |
miker |
jeffdavis: heh ... no, I was just confirming that table inheritance was rediscovered as the reason behind the lack of some fkeys |
| 16:43 |
miker |
plus, "don't delete copy rows for real" |
| 16:47 |
Dyrcona |
Would anyone else be interested in a database function to return ISBN13 for an ISBN? |
| 16:47 |
Dyrcona |
This would be similar to translate_isbn1013 but only one ISBN at a time. |
| 16:47 |
miker |
Dyrcona: sure ... we have an indexing norm... nm :) |
| 16:48 |
miker |
Dyrcona: yes, that would be useful :) |
| 16:48 |
Dyrcona |
I thought that it might be. |
| 16:49 |
Dyrcona |
I have a use case related to the Novelist extracts that JBoyer and I were talking about c. 2:30 EDT. |
| 16:57 |
miker |
Dyrcona++ |
| 17:04 |
Dyrcona |
Well, signing off for now. |
| 17:04 |
Dyrcona |
I should have a branch to share tomorrow morning. |
| 17:06 |
|
mmorgan joined #evergreen |
| 17:16 |
|
mmorgan left #evergreen |
| 18:01 |
|
dbwells_ joined #evergreen |
| 19:35 |
* kmlussier |
likes test plans that end with "the checkout should not explode." |
| 19:35 |
kmlussier |
berick++ |
| 19:47 |
|
bmills joined #evergreen |
| 19:59 |
|
Newziky joined #evergreen |
| 21:22 |
|
brahmina joined #evergreen |
| 21:33 |
|
Christineb joined #evergreen |
| 23:35 |
|
serflog joined #evergreen |
| 23:35 |
|
Topic for #evergreen is now Welcome to the #evergreen library system channel! | We are publicly logged: http://irc.evergreen-ils.org/evergreen | Large pastes at http://paste.evergreen-ils.org |
