| Time |
Nick |
Message |
| 02:07 |
|
stephengwills joined #evergreen |
| 02:33 |
|
sandbergja_ joined #evergreen |
| 06:01 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 07:37 |
|
dbwells joined #evergreen |
| 08:22 |
|
Dyrcona joined #evergreen |
| 08:35 |
|
mmorgan joined #evergreen |
| 08:45 |
Dyrcona |
A propos stephengwills hearing a spider: https://www.youtube.com/watch?v=NrziISO1_xg |
| 08:49 |
mmorgan |
sandbergja++ |
| 08:49 |
mmorgan |
sandbergja++ |
| 08:49 |
mmorgan |
'cuz there's no sandberglja :) |
| 08:49 |
mmorgan |
typos-- |
| 08:56 |
rhamby |
there might be somewhere in the world :) |
| 08:57 |
mmorgan |
rhamby: True! |
| 08:59 |
rhamby |
it's like telling the future, so long as you don't put a time limit on it you're likely to be right |
| 08:59 |
Dyrcona |
Heh. |
| 09:00 |
Dyrcona |
rhamby: Glad you're hear because I encountered something that you *might* have seen before. |
| 09:00 |
Dyrcona |
here, even.... |
| 09:00 |
Dyrcona |
English-- |
| 09:00 |
rhamby |
what's up? |
| 09:01 |
Dyrcona |
DBD::Pg::db do failed: ERROR: index row size 2984 exceeds maximum 2712 for index "authority_full_rec_value_index" while running eg_staged_bib_overlay |
| 09:01 |
Dyrcona |
There's more... |
| 09:01 |
Dyrcona |
I'll pastebin it. |
| 09:03 |
rhamby |
when I've seen that there is a ridiculously long field in the marc being indexed |
| 09:04 |
Dyrcona |
https://pastebin.com/PAvi8zXK |
| 09:05 |
Dyrcona |
That's what I think, but I haven't looked at the record, yet. This happened last night. |
| 09:05 |
rhamby |
I'd wager a decent beer you are right and some tag in there ridiculously long. |
| 09:06 |
rhamby |
s/ridiculously long/ is ridiculously long/ because verbs matter |
| 09:08 |
Dyrcona |
heh. |
| 09:08 |
Dyrcona |
Too bad there's nothing in the output to tell me which record(s) caused this. I'll have look through the input files somehow. |
| 09:08 |
Dyrcona |
rhamby++ |
| 09:09 |
Dyrcona |
to look... Infinitives don't matter. :) |
| 09:09 |
rhamby |
heh |
| 09:14 |
Dyrcona |
I also have a list of record id that give a "Can't call method subfield on an undefined value" message. Maybe it's one or more of these that cause the above error. I'll start with those. |
| 09:29 |
Dyrcona |
Wikipedia referenced in an authority record.... Just throwing that out there, because it's a thing. |
| 09:34 |
gmcharlt |
literary warant is literary warrant |
| 09:34 |
gmcharlt |
or at least headings warrant |
| 09:35 |
Dyrcona |
Uh-huh.... |
| 09:37 |
Dyrcona |
So, none of the records that produce the cant' call method on undefined value error a) look bad, nor b) have a subfield long enough to trigger the index row size error, the longest subfield is 2517 characters long. |
| 09:50 |
Dyrcona |
Looks like they're missing some field defined in authority.control_set_authority_field and related tables, but I haven't determined which ones. |
| 10:28 |
pinesol |
[evergreen|Andrea Buntz Neiman] Docs: correction to 3.7 release notes - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=aabfde8> |
| 11:40 |
pinesol |
[evergreen|Andrea Buntz Neiman] Docs: added script info to Carousels docs - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=467a9bb> |
| 12:27 |
rhamby |
hack-a-way dates: https://www.surveymonkey.com/r/XTH2KYY |
| 12:28 |
rhamby |
^^^ clicky and fill out |
| 12:40 |
jeff |
additional context fromRogan's email: "We are looking at dates in late October and early November and hoping to again meet in person in Indiana. The hotel / conference space has given us three available dates" |
| 12:49 |
|
sandbergja joined #evergreen |
| 13:36 |
Dyrcona |
Well, back to my records issues: There is something wrong with the program that I wrote to check the length of the records' fields/subfields. It didn't find any issues, but just looking at the reject file, I find that the very first record listed has a 670$b that is 5,090 characters long. |
| 14:19 |
rhamby |
my only response to a filed 5,090 long is : yuck |
| 14:20 |
|
tlittle joined #evergreen |
| 14:22 |
Dyrcona |
rhamby: The field appears to be from wikidata.org, so, what I said earlier..... :) |
| 14:22 |
Dyrcona |
I am reprocessing the rest of the file of skipped auths, and it is working so far. |
| 14:23 |
rhamby |
copy and pasting into marc is just evil, but I also have a special disdain for copy and pastes (with various flavors of end of lines) that end up in csv files |
| 14:24 |
Dyrcona |
Yeahp, and particularly with "smart" quotes. |
| 14:25 |
rhamby |
oh yeah, anohter thing to add to my csv correction script .... |
| 14:26 |
Dyrcona |
It's even worse in MARC, since one of them looks like the end of record character. |
| 14:32 |
Dyrcona |
I wonder if we should limit authority_full_rec_value_index to the first 2000 characters or so, using substring? |
| 14:33 |
Dyrcona |
I saw one suggestion online to use GIN for the index, but we can't since the field is not a tsvector. |
| 14:34 |
Dyrcona |
The error output suggests full text indexing as an option: Consider a function index of an MD5 hash of the value, or use full text indexing. |
| 14:55 |
Dyrcona |
Ah ha! I may be able to create a GIN index without changing the field type: https://www.postgresql.org/docs/current/textsearch-tables.html#TEXTSEARCH-TABLES-INDEX |
| 14:59 |
Dyrcona |
I'm not sure that we want a GIN index there, anyway. Limiting the index via substring may be the better option. Oh well. It's something to play around with next week. I suspect GIN may break things without additional code changes. |
| 15:07 |
Dyrcona |
I wonder if authority.authority_full_rec_value_index is even needed. |
| 15:17 |
Dyrcona |
So, I am testing with the following index definition: CREATE INDEX authority_full_rec_value_index ON authority.full_rec (substring(value for 2000)); |
| 16:03 |
|
sandbergja joined #evergreen |
| 16:16 |
* Dyrcona |
calls it a day. Have a nice weekend, everyone! |
| 17:08 |
|
mmorgan left #evergreen |
| 17:41 |
pinesol |
[evergreen|Dan Briem] LP#1920815 Item edit link displayed for all users - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=60ca20e> |
| 17:45 |
pinesol |
[evergreen|Dan Briem] LP#1917495 Angular Catalog: Hold status in View Holds is not saved - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=c6db533> |
| 17:49 |
pinesol |
[evergreen|Terran McCanna] LP1879769 Cannot Edit Carousels from Action Menu - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=04438a0> |
| 17:54 |
* gmcharlt |
claims 1259 |
| 17:59 |
pinesol |
[evergreen|Michele Morgan] LP1905091 Supply environment for added print/email triggers - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=56df766> |
| 17:59 |
pinesol |
[evergreen|Galen Charlton] LP#1905091: stamp DB update - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=422971f> |
| 18:01 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 21:41 |
|
stephengwills left #evergreen |
| 23:36 |
|
sandbergja joined #evergreen |
