| Time |
Nick |
Message |
| 00:59 |
|
jonadab joined #evergreen |
| 00:59 |
|
troy joined #evergreen |
| 00:59 |
|
phasefx joined #evergreen |
| 01:01 |
|
csharp__ joined #evergreen |
| 01:03 |
|
pinesol joined #evergreen |
| 01:05 |
|
gmcharlt joined #evergreen |
| 01:05 |
|
jweston joined #evergreen |
| 01:10 |
|
berick joined #evergreen |
| 06:00 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 07:32 |
|
rjackson_isl_hom joined #evergreen |
| 07:56 |
|
collum joined #evergreen |
| 08:33 |
|
Dyrcona joined #evergreen |
| 08:36 |
|
mmorgan joined #evergreen |
| 08:37 |
|
terranm joined #evergreen |
| 08:38 |
Dyrcona |
mmorgan: That load has run for 3 days 21 hours and 27 minutes so far. |
| 08:40 |
Dyrcona |
I may have to scale back the number of records to load. I think I will do the next run on 9.6 to see if there's much difference with the time from Pg 10. |
| 08:40 |
mmorgan |
Dyrcona: Wow. |
| 08:40 |
Dyrcona |
Suffice it to say that we need to do something about how we handle located URIs. |
| 08:41 |
mmorgan |
Dyrcona++ |
| 08:41 |
mmorgan |
Indeed! |
| 08:41 |
Dyrcona |
This load ran in about 12 hours or so in production last month. I'm going to look and see if I still have the log hanging around. |
| 08:44 |
Dyrcona |
Yeahp: 12 hours 8 minutes 29 seconds. |
| 08:46 |
Dyrcona |
Our production server has about 6x the amount of RAM and has NVMe SSDs rather than an array of spinning rust. |
| 08:47 |
mmorgan |
Not sure I totally trust my math this early on a Monday, but I get 3.5 seconds per record. |
| 08:49 |
mmorgan |
Dyrcona: Which version of Pg do you have on production again? |
| 08:49 |
|
rfrasur joined #evergreen |
| 08:49 |
Dyrcona |
I'm not actually timing the per record times. I removed that feature from the program that does the profiling. I didn't think it would be necessary/interesting with the output from plprofiler. |
| 08:49 |
Dyrcona |
We're running 9.6 in production. I hope to upgrade to Pg 10 in the fall. |
| 08:50 |
* mmorgan |
nods |
| 08:51 |
Dyrcona |
I wonder how much the degraded array affects this. I don't know which array the bad drive is in. |
| 08:58 |
Dyrcona |
mmorgan: That script I shared has a --timing option that uses the Tim::HiRes Perl module to log how long it takes to update or insert each record. |
| 09:00 |
mmorgan |
Thanks! |
| 09:06 |
Dyrcona |
Speaking of the degraded array, it looks like we may have found some replacement drives. I think these are supposed to be hot swap capable. |
| 09:27 |
terranm |
Degraded Array is going to be the name of my autobiography. |
| 09:27 |
Dyrcona |
terranm++ |
| 09:28 |
|
alynn26 joined #evergreen |
| 09:31 |
mmorgan |
Better than Degraded Disarray! |
| 09:33 |
terranm |
That probably is more accurate |
| 09:41 |
|
nfBurton joined #evergreen |
| 10:07 |
|
Christineb joined #evergreen |
| 10:17 |
|
jvwoolf joined #evergreen |
| 10:34 |
|
tlittle joined #evergreen |
| 11:16 |
rfrasur |
tlittle, I just added a comment to the MARC upload bug. It's very possible I'm missing something important though. |
| 11:16 |
tlittle |
rfrasur oh, awesome, thanks! I'll take a look |
| 11:17 |
rfrasur |
https://bugs.launchpad.net/evergreen/+bug/1929749 |
| 11:17 |
pinesol |
Launchpad bug 1929749 in Evergreen "Angularize Acq Load MARC Order Records" [Wishlist,New] |
| 11:21 |
tlittle |
rfrasur Okay, question. At PINES we always use a match set, so I obviously do have a blind spot there. But I was also going from this bug, which said there were problems if you left the matchset blank. Do you guys not use the same settings mentioned in that bug? |
| 11:21 |
tlittle |
https://bugs.launchpad.net/evergreen/+bug/1473143 |
| 11:21 |
pinesol |
Launchpad bug 1473143 in Evergreen "Vandelay Match-Only with blank match set Cause Duplicate records in Acquisitions" [Wishlist,Confirmed] |
| 11:22 |
Dyrcona |
mmorgan: The load finished after 3 days 23 hours and 55 minutes. Just five minutes shy of 4 days. |
| 11:23 |
rfrasur |
Oh, it's possible that we do and have done something to locally fix it (beyond the scope of what I know). But we don't use match sets at all. |
| 11:26 |
rfrasur |
I'm not sure that many are using the "Match-only" merge though other. And it's generally held that it's on the catalogers to do cleanups in terms of possible matches/merges. (This feels like an oversimplification since there are 128 libraries all with their own workflows.) |
| 11:27 |
rfrasur |
Do y'all do centralized cataloging? |
| 11:27 |
rfrasur |
tlittle |
| 11:29 |
mmorgan |
Dyrcona: So I get 27.5 seconds per record, loooong time. |
| 11:29 |
tlittle |
No, not at all. Each system does their own cataloging. They use the Record Match Set and Merge Profiles so that the line items can potentially connect to existing records rather than creating a new acq record for each title, though. |
| 11:29 |
tlittle |
Does not using a match set mean that each of your titles creates an acq record? I don't know if I've ever tried |
| 11:30 |
Dyrcona |
Yes. It could be the server, though. |
| 11:30 |
Dyrcona |
But, I'm about to generate the profiler report, so we'll know where it spent all of its time soon. |
| 11:39 |
Dyrcona |
ERROR: cannot determine the version of the plprofiler extension - please upgrade the database extension to 4.1 or higher. |
| 11:40 |
Dyrcona |
Well, that's not good. AFAICT, I am running the latest version of plgprofiler in the database and the extension is loaded. Google is useless, so far. |
| 11:41 |
rfrasur |
tlittle, not sure. |
| 11:42 |
rfrasur |
And I'm not sure why we didn't go with match sets as required. That was before my time doing much cataloging (and never batch importing). And, as you know, I'm not an acquisitions specialist either (our acq libraries ALL use it differently). |
| 11:45 |
rfrasur |
Just a note about acq admin - I love the tips included in the EDI Account editor. |
| 11:46 |
tlittle |
rfrasur Same! I think those are awesome |
| 11:46 |
Dyrcona |
Right, well, looks like I have to start over..... |
| 11:51 |
mmorgan |
:-( |
| 11:57 |
JBoyer |
I suppose this may be as good a time as any to drop the record count to around 1000 so it doesn’t take ~4 days to test? That should still give plpgprofiler plenty to work with. |
| 12:03 |
|
mmorgan1 joined #evergreen |
| 12:10 |
|
jihpringle joined #evergreen |
| 12:18 |
Dyrcona |
JBoyer: At this point, I might as well wait until the bad drive is replaced and start over with a new data set and a new batch of records. |
| 12:19 |
JBoyer |
Dyrcona++ # good point |
| 12:21 |
Dyrcona |
I'm not so sure that drives I saw this morning are any good. I'll likely find out on Wednesday. |
| 12:38 |
Dyrcona |
So, it turns out that the plprofiler extension installed for Pg 9.6 through Pg 11 was out of date. I just rebuilt it for all the Pg versions on the server: 9.6 to 13, just to be sure. |
| 13:06 |
|
collum joined #evergreen |
| 13:24 |
|
jvwoolf joined #evergreen |
| 14:08 |
|
javier_guel joined #evergreen |
| 14:10 |
javier_guel |
Hi all, I am trying to upgrade from 3.4.2 to 3.7.1 and I am testing upgrade scripts in a dev db, but running 3.5.1-3.6.0-upgrade-db.sql script I am getting the Error "3.5.1-3.6.0-upgrade-db.sql:995: ERROR: syntax error at or near "ON" |
| 14:10 |
javier_guel |
LINE 2: ...S ('au.created', 'au', 'A user was created', 't') ON CONFLIC...", could someone help me? |
| 14:12 |
Dyrcona |
javier_guel: What PostgreSQL version are you running? |
| 14:12 |
javier_guel |
9.6.15 |
| 14:13 |
javier_guel |
Do I need to upgrade to 10v? |
| 14:14 |
Dyrcona |
javier_guel: No. I was about to say that you shouldn't get a syntax error there. |
| 14:14 |
Dyrcona |
Unless something else is wrong with the upgrade script. |
| 14:16 |
csharp__ |
javier_guel: how are you checking the version? |
| 14:16 |
csharp__ |
(PG version) |
| 14:17 |
Dyrcona |
Ah, true. You might be looking at the client version and not the server version. |
| 14:17 |
|
csharp_ joined #evergreen |
| 14:17 |
javier_guel |
opensrf bilblos:~$ psql -V |
| 14:17 |
javier_guel |
psql (PostgreSQL) 9.6.15 |
| 14:18 |
Dyrcona |
javier_guel: That's the client version. You need to know the server version. If you connect to the db, you should see a line like this: psql (12.7 (Ubuntu 12.7-0ubuntu0.20.04.1), server 10.17 (Ubuntu 10.17-1.pgdg20.04+1)) |
| 14:18 |
Dyrcona |
Your version numbers will be different, but that's a quick way to find out the db server version. |
| 14:19 |
csharp_ |
javier_guel: also the output of 'dpkg -l | grep postgresql | grep ^ii' |
| 14:19 |
csharp_ |
(assuming debian/ubuntu) |
| 14:19 |
Dyrcona |
Assuming you run that on the server with the database as well. |
| 14:19 |
javier_guel |
postgresbilblos:~$ psql |
| 14:19 |
javier_guel |
psql (9.6.15, server 9.4.15) |
| 14:19 |
javier_guel |
Type "help" for help. |
| 14:19 |
javier_guel |
postgres=# |
| 14:20 |
csharp_ |
javier_guel: there you go |
| 14:20 |
Dyrcona |
javier_guel: OK, you do probably need to upgrade the server to 9.6. |
| 14:20 |
csharp_ |
javier_guel: upgrade postgresql to at least 9.6 |
| 14:22 |
javier_guel |
perfect, let me try doing that, thanks for your help |
| 14:22 |
javier_guel |
regards from slp, MX |
| 14:23 |
csharp_ |
javier_guel: awesome! welcome and feel free to ask any more questions here if you hit more problems |
| 14:23 |
Dyrcona |
javier_guel: Good luck, and regards from Massachusetts, USA! |
| 14:55 |
|
degraafk joined #evergreen |
| 15:04 |
|
mmorgan1 joined #evergreen |
| 15:50 |
|
nfBurton joined #evergreen |
| 16:18 |
|
rfrasur joined #evergreen |
| 16:33 |
|
jvwoolf left #evergreen |
| 16:49 |
|
mmorgan1 joined #evergreen |
| 17:03 |
|
mmorgan1 left #evergreen |
| 18:00 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 18:41 |
|
jihpringle joined #evergreen |
| 21:22 |
|
Christineb_ joined #evergreen |
| 21:22 |
|
phasefx_ joined #evergreen |
| 21:24 |
|
tsadok joined #evergreen |
