| Time |
Nick |
Message |
| 00:53 |
|
alynn26 joined #evergreen |
| 05:47 |
|
eady joined #evergreen |
| 06:02 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 07:14 |
|
rjackson_isl joined #evergreen |
| 08:15 |
|
collum joined #evergreen |
| 08:36 |
|
mmorgan joined #evergreen |
| 08:40 |
|
mantis joined #evergreen |
| 08:52 |
|
dguarrac joined #evergreen |
| 08:52 |
|
Keith-isl joined #evergreen |
| 08:54 |
|
jvwoolf joined #evergreen |
| 08:59 |
|
Stompro joined #evergreen |
| 09:01 |
|
Stompro joined #evergreen |
| 09:14 |
|
Dyrcona joined #evergreen |
| 09:41 |
Bmagic |
rhamby: marcive in this case, but more of an "in general" question. Is that script open source? I've been using this one to load them: git.esilibrary.com/?p=migration-tools.git;a=blob;f=eg_staged_bib_overlay |
| 09:42 |
rhamby |
that is the same one I use to load though I think I have a few local tweaks that could be moved over (reporting stuff), for the querying I can add it though I probably should clean it up a bit first |
| 09:42 |
rhamby |
I can do that later today though |
| 09:43 |
Bmagic |
I'm happy with the loading script. It's the deletions that I don't have. That's handled with eg_staged_bib_overlay as well? |
| 09:43 |
rhamby |
no, that's a different one |
| 09:44 |
rhamby |
it's a very simple script that uses cstore to find them in the system and very it is only one |
| 09:44 |
rhamby |
it does make the assumption that detection is via the 010 and uses cstore |
| 09:45 |
Bmagic |
I'm looking at introducing the authority deletions feature into: https://github.com/mcoia/mobius_evergreen/blob/master/overdrive/electronic_marc_import.pl |
| 09:52 |
|
Keith-isl joined #evergreen |
| 09:56 |
rhamby |
sounds good. my experience is that the laborous part is identifying them, hence why I scripted that. at least in the files I get I get a big updated of every authority being deleted by LOC but most aren't in use by the library. |
| 10:02 |
Dyrcona |
Bmagic: I use that eg_staged_bib_overlay script, too. |
| 10:22 |
csharp_ |
@hates |
| 10:22 |
pinesol |
csharp_: csharp_ doesn't seem to hate anything. |
| 10:22 |
csharp_ |
@hate too many parallel opensrf requests |
| 10:22 |
pinesol |
csharp_: The operation succeeded. csharp_ hates too many parallel opensrf requests. |
| 10:44 |
Dyrcona |
@hates |
| 10:44 |
pinesol |
Dyrcona hates JavaScript; and Launchpad Search |
| 10:44 |
Dyrcona |
@loves |
| 10:44 |
pinesol |
Dyrcona loves git; sed; OpenBSD; gnu/emacs; git-quickpick; tmux; and #evergreen |
| 10:46 |
|
stephengwills joined #evergreen |
| 11:24 |
|
Keith_isl joined #evergreen |
| 11:49 |
|
jihpringle joined #evergreen |
| 11:49 |
jeffdavis |
We're having lots of problems lately with Vandelay imports being really slow or just hanging, especially when there are multiple imports happening at the same time. We already break down large imports into smaller batches (1000 records or less), are there other tricks we can use to make things run better? |
| 11:59 |
|
Christineb joined #evergreen |
| 12:13 |
jeff |
I've no general vandelay performance recommendations to share, but I'm curious: did "lately" correspond with any changes? |
| 12:16 |
csharp_ |
jeffdavis: in our case the matching parameters slow vandelay import to something like 16s per bib |
| 12:17 |
csharp_ |
so something that speeds up the match set query would help us - last time I looked I wasn't able to pinpoint the bottleneck beyond "something with record attr" |
| 12:21 |
jeffdavis |
I'm seeing "out of shared memory" errors that are similar to bug 1271661 - seems like this would create a bottleneck on match sets? |
| 12:21 |
pinesol |
Launchpad bug 1271661 in Evergreen "record import matching can fail with "shared memory" PostgreSQL errors" [Undecided,Triaged] https://launchpad.net/bugs/1271661 |
| 12:22 |
berick |
i've also seen match set slowness |
| 12:34 |
* jeffdavis |
looks at vandelay.match_set_test_marcxml, backs away slowly |
| 12:34 |
JBoyer |
jeff++ |
| 12:34 |
* JBoyer |
dons hacker hoodie, "you're in" |
| 12:42 |
jeff |
heh |
| 13:20 |
jeffdavis |
From a naive look at that function, it seems like we're using vandelay.get_expr_from_match_set to do too many things: it returns our WHERE clause as a string, and populates the temporary tables as a side-effect. |
| 13:21 |
jeffdavis |
It would be nice if the stuff that goes into the temp tables could just be the return values of a function. |
| 13:21 |
jeffdavis |
Ultimately all we're doing with those temp tables is stashing an array of integers (for qrows) or strings (for jrows). |
| 13:21 |
miker |
jeffdavis: agreed ... at the time, I don't know that it could have been. Now, it could be an hstore array, or some JSON... |
| 14:41 |
|
terranm joined #evergreen |
| 16:50 |
devted |
Has anyone used the Firefox Hatch extension with EG 3.5, 3.6, or 3.7? |
| 17:15 |
|
jvwoolf left #evergreen |
| 17:21 |
|
mmorgan left #evergreen |
| 17:57 |
|
mantis joined #evergreen |
| 18:02 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 18:32 |
|
jihpringle95 joined #evergreen |
