| Time |
Nick |
Message |
| 02:29 |
|
beanjammin joined #evergreen |
| 03:57 |
|
gsams joined #evergreen |
| 07:13 |
|
rjackson_isl joined #evergreen |
| 07:47 |
|
tlittle joined #evergreen |
| 08:08 |
|
Dyrcona joined #evergreen |
| 08:23 |
|
bos20k joined #evergreen |
| 08:34 |
|
jvwoolf joined #evergreen |
| 08:41 |
|
mmorgan joined #evergreen |
| 08:44 |
|
bdljohn joined #evergreen |
| 08:54 |
|
lsach joined #evergreen |
| 09:12 |
|
Dyrcona joined #evergreen |
| 09:13 |
|
yboston joined #evergreen |
| 09:13 |
csharp |
berick: finally getting around to testing your branch for bug 1793802 - I'll let you know how it goes |
| 09:13 |
pinesol |
Launchpad bug 1793802 in Evergreen "Wishlist: Age billing/payment data with circs" [Wishlist,New] https://launchpad.net/bugs/1793802 - Assigned to Bill Erickson (berick) |
| 09:13 |
|
yboston joined #evergreen |
| 09:15 |
|
Dyrcona joined #evergreen |
| 09:30 |
|
Dyrcona joined #evergreen |
| 09:39 |
|
kmlussier joined #evergreen |
| 09:40 |
|
collum joined #evergreen |
| 09:40 |
|
sandbergja joined #evergreen |
| 09:47 |
|
nfburton joined #evergreen |
| 09:55 |
berick |
csharp: heads up, it's still a long-running-ish update. |
| 09:56 |
berick |
2-3 hours IIRC |
| 09:56 |
berick |
will be testing again soon on a large data set |
| 10:02 |
csharp |
2-3 hours is fine compared to the 10 I was seeing without removing/aging any data |
| 10:06 |
|
kipd left #evergreen |
| 10:07 |
|
kipd joined #evergreen |
| 10:10 |
|
AirwaveuF joined #evergreen |
| 10:13 |
|
yboston joined #evergreen |
| 10:18 |
|
mmorgan joined #evergreen |
| 10:26 |
nfburton |
Any reporting gurus in chat today? |
| 10:29 |
csharp |
nfburton: I may be able to help (not claiming the "guru" title, but I'm pretty experienced :-) ) |
| 10:31 |
nfburton |
LOL okay. I have been running reports for my catalogers but the one thing I can't seem to search by is Nature of Contents. |
| 10:31 |
csharp |
is that a marc field? |
| 10:32 |
nfburton |
It is 008(24-27). I can get Form of Item which is right beside it at (23) but cant seem to find Nature of Contents |
| 10:32 |
csharp |
hmm - lemme look at some MARC docs |
| 10:33 |
nfburton |
My last ditch effort is a manual SQL query to create a report for them. |
| 10:35 |
csharp |
doesn't look like we break that field out in a reporter friendly way |
| 10:37 |
nfburton |
Okay. I will have to run an xpath query then. Thought I may be just missing something. Thanks |
| 10:37 |
rhamby |
nfburton: there may be an easier path than xpath (so to speak) what are you trying to get the MARC based material format? |
| 10:37 |
csharp |
I've been able to go my entire 10+ years supporting Evergreen without needing that though |
| 10:37 |
|
khuckins_ joined #evergreen |
| 10:38 |
csharp |
wondering if there's another way to get what they're after? |
| 10:38 |
csharp |
I've got several reports using form, type, and audn, but I've not had a request for cont |
| 10:38 |
rhamby |
csharp: I suspect so, a lot of that type of data gets calculated into evergreen values in various ways |
| 10:39 |
csharp |
yeah |
| 10:40 |
csharp |
I do see entries for "cont1" in metabib.record_attr, though, so it may just not be exposed |
| 10:41 |
csharp |
most are " " |
| 10:41 |
nfburton |
I can see uses for other identifiers in that field but I am trying to find all my Graphic Novels. The MARCs aren't super consistent so I am trying to blanket as much as possible |
| 10:41 |
csharp |
ah |
| 10:44 |
jeff |
dickreckard: your messages to the channel last night wouldn't have been seen by most, because you weren't voiced. i can repeat them now so that more folk here can see and respond. |
| 10:44 |
jeff |
<dickreckard> hello |
| 10:44 |
jeff |
<dickreckard> i have an issue when using non default ports for the ssl web admin interface.. (i am using an nginx proxy that forwards to port 7443).. |
| 10:45 |
jeff |
<dickreckard> the perl authenticator seem to causes an infinite 302 redirect loop.. did i do something wrong or just found a bug? |
| 10:58 |
rhamby |
nfburton: if you're looking to ID graphic novels by marc identifiers I suspect you'll be disappointed unless your catalogers are shockingly consistent and detailed |
| 10:58 |
berick |
dickreckard: bug 1778728 |
| 10:58 |
pinesol |
Launchpad bug 1778728 in Evergreen "staff client "too many" redirects" error when using nginx proxy" [Undecided,Incomplete] https://launchpad.net/bugs/1778728 |
| 11:00 |
nfburton |
rhamby: Yeah, that was my feelings about it. I can look up comics in the Literary Form but that was the old method and only some have the 6 in the 008 field. |
| 11:01 |
nfburton |
Its a chicken chase |
| 11:01 |
|
mmorgan1 joined #evergreen |
| 11:02 |
rhamby |
nfburton: even pre-RDA those fields were rarely filled out consistently by libraries, often because catalogers worked at institutions that didn't use it and it wasn't exposed in any way, then it got copy cataloged and ... etc... |
| 11:02 |
nfburton |
lol way to think of the future eh? |
| 11:03 |
rhamby |
nfburton: are they any conventions your catalogers use for graphic novel call numbers like 'GN'? (some places do similar things) |
| 11:03 |
Dyrcona |
People don't think of the future. |
| 11:03 |
nfburton |
Ah well, I've been on the monumental task of trying to clean up our catalog. I'm hoping we get MARCive or a similar company to fix our MARCs |
| 11:03 |
rhamby |
Dyrcona: and that is probably the truest statement we can make |
| 11:04 |
nfburton |
Yeah, we started doing that with Call Number Prefix but any pre-standard are unidentified |
| 11:04 |
nfburton |
MARC is super outdated anyway |
| 11:04 |
Dyrcona |
Indeed it is. Do you have a better solution? :) |
| 11:05 |
rhamby |
nfburton: any chance catalogers were consistent about a 650 $a for 'Graphic Novels'? |
| 11:05 |
rhamby |
nfburton: think you'd have better chances with the 650 than the fixed fields |
| 11:07 |
nfburton |
I will give it a try. Thanks |
| 11:07 |
nfburton |
Dyrcona: I've heard about BIBFRAME but unsure how that is playing out |
| 11:08 |
rhamby |
nfburton: you might also find 655s of 'Graphic novels' |
| 11:09 |
rhamby |
nfburton: and if you mainly get comic book based ones looking for 264$b s including 'DC' 'Marvel' and 'Dark Horse' 'Viz' and other publishers might get you close |
| 11:09 |
rhamby |
nfburton: now that I'm thinking about it I'd probably start with the 264$b |
| 11:10 |
* rhamby |
says that it takes a village to build a MARC record. |
| 11:11 |
csharp |
@quote add <rhamby> it takes a village to build a MARC record. |
| 11:11 |
pinesol |
csharp: The operation succeeded. Quote #191 added. |
| 11:12 |
csharp |
berick: upgrade script for bug 1793802 is failing because there are multiple duplicate IDs in money.payment_view - did you encounter anything like that? |
| 11:12 |
pinesol |
Launchpad bug 1793802 in Evergreen "Wishlist: Age billing/payment data with circs" [Wishlist,New] https://launchpad.net/bugs/1793802 - Assigned to Bill Erickson (berick) |
| 11:13 |
csharp |
also, surprised that's possible given the view definition |
| 11:14 |
csharp |
oh - this is the "one payment can apply to multiple bills" problem :-/ |
| 11:15 |
csharp |
hmm - "payment_pkey" PRIMARY KEY, btree (id) - that doesn't prevent duplicate PK ids? |
| 11:16 |
berick |
csharp: that's unexpected |
| 11:18 |
csharp |
psql:XXXX.schema.aged-billing-payment.sql:15: ERROR: duplicate key value violates unique constraint "aged_payment_pkey" <-- the actual error |
| 11:18 |
Dyrcona |
"Inconceivable!" |
| 11:18 |
berick |
csharp: yeah, i'm unsure how you could get dupe IDs, all payments should share 'money.payment_id_seq' |
| 11:19 |
Dyrcona |
What if the script tries to age the same payment twice? Just spitballing.... |
| 11:19 |
|
beanjammin joined #evergreen |
| 11:20 |
berick |
Dyrcona: good question, csharp can you check if you actually have dupe ID in payment_view? |
| 11:20 |
csharp |
yeah - the dupes are present in the view *and* the money.payment table |
| 11:21 |
Dyrcona |
Well, that sounds like a hosed schema to me. Good luck with that! |
| 11:21 |
berick |
heh |
| 11:21 |
csharp |
https://pastebin.com/i3F96FuW is the one that it failed on, but there are many more |
| 11:22 |
berick |
csharp: are they different payment types? |
| 11:22 |
csharp |
yes |
| 11:22 |
berick |
if you \d the tables for the 2 types, they both show: |
| 11:22 |
csharp |
https://pastebin.com/3U793cNd - the view version |
| 11:22 |
berick |
id | bigint | not null default nextval('money.payment_id_seq'::regclass) |
| 11:23 |
csharp |
yes |
| 11:23 |
berick |
hm, but each its own *_payment_pkey |
| 11:23 |
berick |
primary key definition |
| 11:24 |
csharp |
"forgive_payment_pkey" PRIMARY KEY, btree (id) and "cash_payment_pkey" PRIMARY KEY, btree (id) |
| 11:24 |
berick |
yeah |
| 11:24 |
berick |
which suggests to me they are not guaranteed to be unique |
| 11:24 |
berick |
unlikely, given the sequence, but not guaranteed |
| 11:25 |
berick |
well, though, i though PG always handed out new values for calls to NEXTVAL() |
| 11:27 |
berick |
csharp: are they all dated 2010? |
| 11:27 |
berick |
could be migration funk |
| 11:27 |
csharp |
about to check that out |
| 11:27 |
csharp |
yeah :-/ |
| 11:27 |
csharp |
the good news is that there are only 738 IDs with dupes |
| 11:28 |
Dyrcona |
The bad news is there are 738 IDs with dupes. |
| 11:29 |
berick |
Dyrcona's glass is always half full :) |
| 11:31 |
csharp |
here's the date distribution of the dupes: https://pastebin.com/Y2ADpRaH |
| 11:31 |
Dyrcona |
And, they're for different payment types? |
| 11:32 |
|
sandbergja joined #evergreen |
| 11:33 |
csharp |
Dyrcona: correct |
| 11:33 |
csharp |
ohhhh |
| 11:33 |
csharp |
I remember this |
| 11:34 |
csharp |
this was a period where the DB was totally screwed because of some corner-case C library bug in postgres |
| 11:35 |
csharp |
DB would slow to a crawl and the libraries were forced offline |
| 11:35 |
csharp |
miker may remember :-) |
| 11:35 |
csharp |
spring 2010 |
| 11:36 |
Dyrcona |
Pg 8.{somthing}? |
| 11:36 |
csharp |
yeah |
| 11:36 |
csharp |
probably 8.1 or 8.3 |
| 11:37 |
* csharp |
shudders from the memory of it |
| 11:39 |
* csharp |
ponders a fix |
| 11:39 |
dickreckard |
ah thanks berick ! :) |
| 11:40 |
dickreckard |
still havent learned how to search for bugs apparently.. |
| 11:41 |
Dyrcona |
dickreckard: It's not you. The search on Launchpad is pretty lousy. |
| 11:41 |
kmlussier |
dickreckard: I've been working on Evergreen for more than eight years and still have trouble finding bugs. I would be more surprised if you always found the bugs you're looking for. :) |
| 11:41 |
dickreckard |
heheh |
| 11:41 |
Dyrcona |
I sometimes try filing a new bug. That often finds the one that I'm looking for. |
| 11:42 |
dickreckard |
haha that's the more efficient search |
| 11:42 |
dickreckard |
duplicating |
| 11:42 |
Dyrcona |
Saving the bug emails and searching those is often better, too. |
| 11:42 |
* Dyrcona |
gets emails on all the bugs. |
| 11:44 |
* nfburton |
waves hand attempting Jedi mind trick. "This is not the bug you are looking for" |
| 11:44 |
dickreckard |
another question.. does anybody have a clue where to look if a marc batch import fails silently ending up in an empty queue, with no errors in the osrfsys log? |
| 11:45 |
Dyrcona |
dickreckard: Database logs, maybe, but db errors usually show up in Evergreen logs, too. Look for "severe query error" or something like that in your logs. |
| 11:46 |
dickreckard |
cool, thanks. maybe they don't because the db is on another server |
| 11:46 |
Dyrcona |
Maybe "attempt to update the database failed"? |
| 11:47 |
Dyrcona |
Nah, they still will, at least the Evergreen part of the message will show up. |
| 11:47 |
* Dyrcona |
looks for an example. |
| 11:47 |
|
jvwoolf1 joined #evergreen |
| 11:50 |
Dyrcona |
I see a bunch that have DATABASE_QUERY_FAILED in my error and warning logs. Looking for that string might help locate an issue. |
| 11:54 |
dickreckard |
thanks i'll check. but i get no stderr log entries when that happens.. but i just saw something weird maybe.. open-ils.auth logs at GMT, while the rest at GMT+2 |
| 11:54 |
dickreckard |
looks like some internal timezone disagreement |
| 11:54 |
csharp |
ugh - I can't just delete all the rows with dupe IDs because that would throw off the balances - the payments themselves are valid even though the IDs are wrong |
| 11:55 |
csharp |
does it make sense to assign duplicated rows new IDs from nextval? |
| 11:58 |
Dyrcona |
csharp: It probably does. |
| 11:59 |
Dyrcona |
dickreckard: Those errors will appear in gateway.log or osrfsys.log. I have a complicated log configuration using rsyslog on a remote server. |
| 11:59 |
Dyrcona |
csharp: The trick will be updating the correct billings, I'd imagine. |
| 12:05 |
|
jihpringle joined #evergreen |
| 12:06 |
mmorgan |
Hm. There's no direct reference from payments to billings, is there? |
| 12:06 |
mmorgan |
It all links through the xact. |
| 12:09 |
|
beanjammin joined #evergreen |
| 12:14 |
Dyrcona |
yeahp. |
| 12:17 |
csharp |
yeah - theoretically the payment IDs should matter from the perspective of the xact |
| 12:18 |
csharp |
s/should/shouldn't/ |
| 12:26 |
berick |
csharp: I think money.account_adjustment is the only thing to be mindful of |
| 12:26 |
berick |
since it references money.billing (id) |
| 12:26 |
berick |
wait, nevermind |
| 12:27 |
berick |
payments, not billings |
| 12:27 |
csharp |
right - thanks for checking for me though :-) |
| 12:56 |
Dyrcona |
Is there a way to tell active from passive event definitions in the database? |
| 12:57 |
berick |
Dyrcona: see the hook |
| 12:58 |
Dyrcona |
berick: thanks! |
| 12:59 |
Dyrcona |
For the logs: action_trigger.hook.passive (boolean). |
| 13:00 |
Dyrcona |
berick: Another question while you're here. Only passive hooks need a delay, max_delay, and a retention interval to get purged? Do active hooks need anything special? |
| 13:00 |
Dyrcona |
Just retention interval, I'm guessing. |
| 13:00 |
berick |
right, just retention interval |
| 13:01 |
berick |
... for active hooks |
| 13:03 |
Dyrcona |
Thanks muchly! berick++ |
| 13:06 |
Dyrcona |
I assume it makes no sense for a retention interval to be negative, even for events with negative delays. |
| 13:06 |
berick |
correct |
| 13:07 |
berick |
it's based on the last edit date of the event (among other things), which will never be in the future. |
| 13:07 |
Dyrcona |
Thanks, again. |
| 13:08 |
|
nfburton joined #evergreen |
| 13:12 |
|
khuckins_ joined #evergreen |
| 13:21 |
plux |
I have a sticky issue on a 3.2.0 migration. All the updates and pingest appeared to run fine but webby will not return title in any record summary views. |
| 13:21 |
plux |
I’m assuming something got missed in db upgrades along the way but starting to wonder if it’s a template/angular issue |
| 14:02 |
Dyrcona |
plux: I think the silence means that no one else has encountered that, yet. |
| 14:05 |
plux |
it’s quite odd. I don’t see anything to suggest a bug so I’m more inclined to think something didn’t update properly in the db upgrades. There’s no real useful errors I’m seeing although JS console does see that someting is not returning. The link builds fine but nothing returns to screen. |
| 14:07 |
Dyrcona |
plux: Is it just the title field that's missing or is the whole record summary blank? |
| 14:10 |
plux |
just title…all the other fields populate fine |
| 14:15 |
Dyrcona |
Maybe a template is busted, or some metabib field isn't populated. I'd the summary.tt2 template first. (I think that's the one you want to look at.) |
| 14:18 |
plux |
I’ll do some more template digging…. |
| 14:25 |
gmcharlt |
plux: one possibility is that the title display field may be interacting badly with a customized config.metabib_field defintion |
| 14:26 |
gmcharlt |
I'd be curious to see whether metabib.display_entry is contains title statements for your records |
| 14:27 |
plux |
I’ll check display_entry…..I’ve been through metabib_field several times. The only thing that was changed thee was resetting all the mods32 to mods33 in metabib_field |
| 14:31 |
csharp |
for the logs, here's how I solved my dupe ID issue earlier: https://pastebin.com/mCPZVCM2 |
| 14:31 |
csharp |
took a long time for me to get there, but it's pretty simple in the end :-) |
| 14:32 |
berick |
csharp++ |
| 14:34 |
|
mmorgan1 joined #evergreen |
| 14:34 |
plux |
checked a few records in metabib.display_entry and they all look fine for title |
| 14:41 |
csharp |
plux: anything in the postgres logs? |
| 14:50 |
plux |
no, not for this |
| 15:08 |
jeff |
plux: do you have a value for title in reporter.materialized_simple_record for the records in question? |
| 15:12 |
jeff |
plux: also, can you share the output of the following query? SELECT cdfm.*, cmf.field_class, cmf.name FROM config.display_field_map AS cdfm LEFT JOIN config.metabib_field AS cmf ON cdfm.field = cmf.id WHERE cdfm.name = 'title'; |
| 15:14 |
plux |
"title"6false"author""corporate" |
| 15:15 |
plux |
I would have expected field_class to be title here? and the field to be 5 as that’s what it is in metabib.display_field |
| 15:18 |
|
mmorgan joined #evergreen |
| 15:23 |
jeff |
plux: yup, you hit something we hit. |
| 15:23 |
jeff |
plux: the issue is that the upgrade script assumes that certain config.metabib_field IDs will be certain fields, and our (and now your) database didn't meet those assumptions. |
| 15:24 |
plux |
ah yes and reporter.materialized_simple_record for those ids the title field is indeed empty |
| 15:24 |
jeff |
amusingly, i suspect that in some cases where an item has a corporate author, you'll have a title that is the corporate author. :-) |
| 15:24 |
jeff |
(at least, we saw some of that) |
| 15:25 |
plux |
I’ll check for that…..so we’ll have to fix config.metabib_field and reingest |
| 15:26 |
jeff |
yes, that's what we did. i'll open a bug to see if we can either make the upgrade script more robust or at least put a warning in the release notes. |
| 15:26 |
jeff |
plux: what version of Evergreen were you running prior to this upgrade? |
| 15:26 |
plux |
3.0.3 |
| 15:26 |
jeff |
we encountered the issue when we upgraded to 3.1, so i'm guessing you... yep, were not previously on 3.1 |
| 15:27 |
plux |
yes. I had tried a 3.1 upgrade previously and I think the issue was there as well but we abandoned that to wait for 3.2 |
| 15:28 |
plux |
I have to go actually but this is a great help - I don’t think I would have caught that |
| 15:28 |
plux |
I’ll keep an eye out for a bug report and try to backtrack the proper fix from there |
| 15:31 |
jeff |
the 3.1 upgrade script expects config.metabib_field ID 6 to be title/proper. |
| 15:31 |
jeff |
er. |
| 15:31 |
jeff |
lost plux. :-) |
| 15:34 |
|
khuckins joined #evergreen |
| 17:07 |
|
khuckins_ joined #evergreen |
| 17:07 |
|
mmorgan left #evergreen |
| 17:48 |
|
khuckins joined #evergreen |
| 18:32 |
|
sandbergja_ joined #evergreen |
| 19:04 |
|
Dyrcona joined #evergreen |
| 20:57 |
|
beanjammin joined #evergreen |
