| Time |
Nick |
Message |
| 06:41 |
|
jonadab joined #evergreen |
| 07:26 |
|
kworstell-isl joined #evergreen |
| 07:41 |
|
redavis joined #evergreen |
| 08:07 |
|
collum joined #evergreen |
| 08:11 |
|
BDorsey joined #evergreen |
| 08:45 |
|
mmorgan joined #evergreen |
| 08:46 |
|
dguarrac joined #evergreen |
| 08:59 |
|
Dyrcona joined #evergreen |
| 09:05 |
|
mantis1 joined #evergreen |
| 09:32 |
|
mmorgan1 joined #evergreen |
| 10:16 |
Dyrcona |
Oof... using XML::LibXML to parse, modify, and write out the fm_IDL.xml changes more than I'd like. Looks some extra whitespace gets removed, even when I set the "format" to 0. |
| 10:19 |
Dyrcona |
I am writing to a separate file and comparing for now, not overwriting the original. I planned to use the Emacs emerge feature to merge the changed lines into the original, so I can skip the extraneous changes, but their presence makes that harder. Besides, I do not disagree with all of the changes, either.... |
| 10:20 |
Bmagic |
Dyrcona: Warning from bibliographic record 498971: no mapping found at position 3 in R. |
| 10:20 |
Bmagic |
Warning from bibliographic record 546406: Use of uninitialized value in join or string at /usr/share/perl5/MARC/Field.pm line 696 |
| 10:21 |
Dyrcona |
Bmagic: You probably have some bad records. It happens. I've seen records where the character set issues are so bad, "random" stuff is being used as a tag. |
| 10:21 |
Bmagic |
totally |
| 10:21 |
Dyrcona |
I don't think that has anything to do with the Pg version. |
| 10:21 |
Bmagic |
I agree |
| 10:23 |
Dyrcona |
On the plus side, my program to identify and update the IDL for required fields worked, and most likely saved me hours of work compared to looking through the schema and updating them fields manualy. |
| 10:23 |
Bmagic |
I'd like to add that marc_export is slower than our in-house extractor. By a factor of 15 |
| 10:23 |
Bmagic |
(not including items) |
| 10:24 |
Dyrcona |
Bmagc: What Evergreen version? There are patches in main/3.12 to speed marc_export up. I backported them to CW MARS production. |
| 10:24 |
Bmagic |
those speedups were for attached items? EG 3.11.1 |
| 10:24 |
Dyrcona |
Stompro++ # For most of the performance improvements. |
| 10:25 |
Bmagic |
I was tracking that thread |
| 10:25 |
Dyrcona |
Mostly for items, yes. |
| 10:26 |
Bmagic |
this execution is sans items. At least I think it is. I didn't pass any arguments to marc_export. My command: cat records.txt | marc_export > test.mrc |
| 10:27 |
Dyrcona |
Well, use your own extractor if it is faster. Also, "Patches welcome!" |
| 10:27 |
pinesol |
News from commits: LP1839364 Move login page error message; add ARIA <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=8a92c0619f366e1854575d686c3619d5d559f694> |
| 10:27 |
pinesol |
News from commits: LP#1945003: (follow-up) redo lint fixups <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=4fdaf17cf6a91a6f7e3a8bcf1864aef526ffeed8> |
| 10:27 |
pinesol |
News from commits: LP1945003 Vand. session tracker serialize pcrud calls <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=182f43a397ee5109dd25359b60097f89efa91c8f> |
| 10:27 |
pinesol |
News from commits: LP#1945003: (prep) temporarily undo lint fixups <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=0c1bf11e407938fdb5216e2c0328d8169584724e> |
| 10:27 |
pinesol |
News from commits: LP#2049654: fix Expand All button icon on purchase order page <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=9d870f2515980a31781fa68b555808a0dfa2bbd0> |
| 10:27 |
pinesol |
News from commits: LP#1980273: ensure patron alerts are displayed in AngularJS patron summary <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=ebf164d1c9bc6588ece9eda9cd4eea640269d67d> |
| 10:27 |
pinesol |
News from commits: LP20474587 Fix copy stat cat fleshing in SuperCat <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=20afec33df906b62c908dd19c580a873f9bb4043> |
| 10:27 |
pinesol |
News from commits: LP2047714 - Fix tab defaults in Traditional catalog <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=c8b3dea9772d91f83f72389f959f6967af6e9390> |
| 10:27 |
* Dyrcona |
grumbles about having to rebase for the second time today..... |
| 10:30 |
Dyrcona |
543 differeneces... |
| 10:30 |
Dyrcona |
differences... |
| 10:32 |
Dyrcona |
Interestingly, XML::LibXML preserves the comments, etc., from the original document. That kind of surprised me. I expected to only get the content of the actual IDL node and not the preceding and trailing comments. |
| 10:33 |
Dyrcona |
All right. I'll give the merge a shot. |
| 10:33 |
|
sandbergja joined #evergreen |
| 10:34 |
Dyrcona |
huh. merge says 962 differences. but diff -u | grep -c '^@@' said 543.. |
| 10:36 |
Dyrcona |
Oof. gotta split a difference. Think I'm going to have to do a lot of that. |
| 10:36 |
|
jvwoolf joined #evergreen |
| 10:38 |
Stompro |
bmagic, Most of the time being used in marc_export when not dealing with items is parsing and converting in libraries, I couldn't see much I could improve there. Is your extractor multi-threaded? |
| 10:39 |
Bmagic |
yeah, it's multi threaded, but because of the Encode.pm issue with multi-thread, I spawn new processes manually, each dumping their portion of the extraction to disk, and the main process scoops up the output and stitches them back into the main file output |
| 10:40 |
Dyrcona |
It's not multithreaded. It's multiple processes. We're programmers, can we use proper terminology? |
| 10:40 |
Bmagic |
hehe, ok, yes "multiple processed" |
| 10:40 |
Dyrcona |
You can't use Perl threads with the Encode module, so there's no way it's multithreaded. |
| 10:41 |
Bmagic |
right, can't use multithreading |
| 10:41 |
Stompro |
I looked at multi-threading for a bit, and then Dyrcona warned me that it won't work. |
| 10:42 |
Stompro |
How many processes are you using when it is 15x faster? |
| 10:42 |
Dyrcona |
Basically, never use threads in Perl. |
| 10:43 |
Bmagic |
Stompro: in this example, eight |
| 10:45 |
Stompro |
I was planning on looking at how to turn marc_export into a multi process exporter... but I'm only a part time programmer, not sure when I'll get back to that. |
| 10:46 |
Dyrcona |
marc_export would be faster if your run multiple instances. Just split your ID file up and run multiple exports, then splice them together at the end. |
| 10:46 |
Dyrcona |
you can just cram binary MARC files together. XML would require a bit more work. |
| 10:47 |
Bmagic |
Dyrcona: yes, that comes to mind. Splitting the ID's then running 8 processes of marc_export. Then we'd need to combine the outputs at the end. My extractor is here: https://github.com/mcoia/mobius_evergreen/tree/master/bib_extract |
| 10:47 |
Bmagic |
be gentle, that code is old and junky |
| 10:48 |
Bmagic |
It keeps working for me, so I've not prioritized it's rewrite |
| 10:48 |
Dyrcona |
Bmagic: I've looked at it before, I think. That's fine if it works. |
| 10:50 |
Stompro |
We are using OAI-PMH for our main export of bibs now, so I don't really use marc_export currently. OAI-PMH works great with some adjustments to allow record buckets to be the source. |
| 10:51 |
Bmagic |
my exporter takes a supplied query, located in the queries.txt file. It can be anything, including selecting ID's from a record bucket |
| 10:56 |
* Dyrcona |
is not splitting all of the differences that I should. I'm letting some whitespace changes in if they're accompanied by add the required attribute. |
| 11:14 |
Dyrcona |
I guess I should have skipped views... i.e. oils_persist:readonly='true' |
| 11:36 |
|
jvwoolf joined #evergreen |
| 11:50 |
mantis1 |
lol this is a funny one |
| 11:51 |
mantis1 |
is it Authority Browse Axis or Authority Browse Axes? |
| 11:51 |
mantis1 |
working on this: https://bugs.launchpad.net/evergreen/+bug/1948991 |
| 11:51 |
pinesol |
Launchpad bug 1948991 in Evergreen "Administration Options - Mismatch between menu name and interface name" [Medium,Confirmed] - Assigned to Gina Monti (gmonti90) |
| 11:57 |
pinesol |
News from commits: LP#1889133: (follow-up) fix multiSortable <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=c03084b9c13955a26885745246e68ed75d2db1bb> |
| 11:57 |
pinesol |
News from commits: LP1889133 Follow Up <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=4bb742c66cab65c9ff18531ca98b8a6a6a9dc948> |
| 12:04 |
|
jihpringle joined #evergreen |
| 12:33 |
Dyrcona |
I think I'm going to do this again and make sure to skip read-only classes, as dumb as that sounds. |
| 12:39 |
Dyrcona |
Think I'll be more picky with the merge, too. |
| 12:39 |
Dyrcona |
Anyway, after lunch. |
| 13:08 |
|
sandbergja joined #evergreen |
| 13:19 |
|
sandbergja joined #evergreen |
| 13:45 |
|
jvwoolf joined #evergreen |
| 14:26 |
|
kco joined #evergreen |
| 14:31 |
|
kco joined #evergreen |
| 14:38 |
|
jihpringle joined #evergreen |
| 15:20 |
|
kmlussier joined #evergreen |
| 15:30 |
|
mantis1 left #evergreen |
| 15:31 |
|
jvwoolf joined #evergreen |
| 15:32 |
|
jvwoolf joined #evergreen |
| 15:38 |
|
jvwoolf left #evergreen |
| 16:51 |
* Dyrcona |
is starting to think I should just replace the whole fm_IDL.xml instead of just merging certain lines. It would be a lot faster. |
| 16:51 |
Dyrcona |
Plus, the spacing would be more consistent. |
| 16:51 |
Bmagic |
I like this plan |
| 16:52 |
Dyrcona |
Also the puncutation would all be " instead of some " and some '. |
| 16:53 |
Dyrcona |
Multiline entries get squashed to 1 line. |
| 16:54 |
Dyrcona |
i should say elements that span more than 1 line now have line endings removed. |
| 16:55 |
kmlussier |
I don't think I will ever lose the habit of saying copy location groups instead of shelving location groups. I spend way too much time correction that one word in my communications. |
| 16:58 |
Dyrcona |
Bmagix: In that case, I think I'll just modify my program to replace fm_IDl.xml tomorrow and do it again. |
| 16:58 |
* Dyrcona |
signs out! |
| 16:59 |
Dyrcona |
Good evening, everyone! |
| 17:03 |
|
mmorgan left #evergreen |
| 18:15 |
|
kmlussier left #evergreen |
