| Time |
Nick |
Message |
| 04:08 |
|
JBoyer joined #evergreen |
| 06:00 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 07:33 |
|
rjackson_isl_hom joined #evergreen |
| 07:48 |
|
collum joined #evergreen |
| 08:19 |
|
rfrasur joined #evergreen |
| 08:21 |
|
Dyrcona joined #evergreen |
| 09:00 |
|
mantis joined #evergreen |
| 09:04 |
Dyrcona |
I'm working on a program to read a list of copy barcodes from a spreadsheet, look them up in Evergren, check them in, and then delete the copy. |
| 09:09 |
Dyrcona |
I have a spreadsheet with 2,779 barcodes. Of these., 324 appear to be deleted already.. https://pastebin.com/H9A0RYwU should output "Total Barcodes: 2779\nBarcodes Found: 2459". Instead, the latter number is always 499. |
| 09:11 |
Dyrcona |
I've also tried variations where the cstore session is created outside the loop, I've tried searching with a CStoreEditor. The result is always 499. A database query for non-deleted copies with the barcode list returns 2459. |
| 09:11 |
Dyrcona |
The question for the OpenSRF experts is, what am I not doing/doing wrong? |
| 09:14 |
Dyrcona |
When I run the full program to do the deletions, it reports the same 499 found, deleted, etc. Subsequent runs fail to find any copies. |
| 09:18 |
Dyrcona |
I'll try a version no using Cronscript.pm to see if that makes a difference. |
| 09:21 |
Dyrcona |
And, no, I don't think I will. That would be too much effort for this. :) |
| 09:28 |
Dyrcona |
Ok. I get the correct numbers in production, so it is something in my development environment. |
| 09:29 |
Dyrcona |
I also switched to using Apputils->simplereq rather than manually creating the session. |
| 09:30 |
Dyrcona |
That second version still only gives 499 on my development server. |
| 09:34 |
Dyrcona |
So, what setting should I look at? |
| 09:35 |
Dyrcona |
Hmm... git diff says my opensrf.xml.example files are the same. |
| 09:39 |
Dyrcona |
It's not a data difference, either. The development server uses a copy of production, and the lookup SQL gives the same number of not-deleted copies as in production: 2,455. |
| 09:41 |
|
jvwoolf joined #evergreen |
| 09:46 |
Dyrcona |
2021-12-09 09:16:58.597 [info] <0.2465.0>@ejabberd_c2s:process_terminated:271 (tcp|<0.2465.0>) Closing c2s session for opensrf jasontest.private/client_at_jasontest.cwmars.org_20184: Connection failed: connection closed |
| 09:47 |
Dyrcona |
Maybe I missed an ejabberd setting. Bet it's the rate limiter/whatever. |
| 09:58 |
csharp_ |
Dyrcona: I'm seeing tons of those messages in my ejabberd.log |
| 10:04 |
Dyrcona |
Yeah. I looked it up online, and it's a nonissue. I've determined it is not an ejabberd problem on my development vm. |
| 10:04 |
Dyrcona |
The behavior suggests a data problem, but I've checked the database so many times, unless it's the number of connections allowed.... |
| 10:05 |
jeff |
Have the program sleep when it gets to 499 - 1 and look at the system state? |
| 10:06 |
jeff |
If you're not interacting with anything that passes through Apache, then it's probably not the recommended "MaxConnectionsPerChild 500" config setting there. |
| 10:07 |
jeff |
But I wonder if you're creating a new *something* (connection/request object/session/etc) for each query and not closing/destroying/reusing it. |
| 10:09 |
Dyrcona |
jeff: I'll try the sleep if upping the Pg connections doesn't work. It's not using Apache. It's Perl. If you look at the script that I pasted, it finishes and disconnects. Apputils' simplereq disconnects when it is done. It works in production, so it's got to be a configuration or resource issue. |
| 10:11 |
Dyrcona |
Pg max_connections made no difference. |
| 10:13 |
Dyrcona |
Weird...It seems to skip the first couple of thousand barcodes. |
| 10:17 |
Dyrcona |
Remove the uniq and sort, it still does only 499, but it starts finding them earlier.... The query to look up the copies by barcode still says there are 2,455. |
| 10:18 |
Dyrcona |
I think I'll wipe out the /openils directory and the perl modules and reinstall. I did a make install yesterday, but maybe there's some junk from a different eg version hanging around. |
| 10:18 |
Dyrcona |
Failing that, I was going to build a new VM, so what's another? |
| 10:28 |
Dyrcona |
Now, the script can't authenticate, says no seed found. |
| 10:29 |
Dyrcona |
services say they're running. |
| 10:30 |
Dyrcona |
Oh, right. need to fix the memcached settings. :) |
| 10:34 |
Dyrcona |
And, I'm back where I started.... |
| 10:34 |
Dyrcona |
"Nuke it from orbit. It's the only way to be sure." |
| 10:35 |
Dyrcona |
@blame Excel/xlsx |
| 10:35 |
pinesol |
Dyrcona: Excel/xlsx WILL PERISH UNDER MAXIMUM DELETION! DELETE. DELETE. DELETE! |
| 10:36 |
Dyrcona |
Looking at the log more carefully, I see this: Dec 9 10:34:05 jasontest open-ils.cstore: [INFO:6887:osrf_application.c:1075:] CALL: open-ils.cstore open-ils.cstore.direct.asset.copy.search {"barcode":"3.0453000494612E13","deleted":"f"},{"flesh_fields":{"acp":["circulations"]},"flesh":1} |
| 10:37 |
jeff |
aha! |
| 10:37 |
Dyrcona |
The barcode is a "number" according to that output. I guess only the barcodes that begin with the letter A are working. Now, why this happens on the test vm and not in production... Hm. I edited the sheet in LibreOffice this morning. Maybe that "fixed" it? |
| 10:38 |
jeff |
(now some fun is to be had with determining if the text->number type confusion happened at log time or before.) |
| 10:39 |
Dyrcona |
jeff: It's before. Excel does that with barcodes. I think the spreadsheet I used in production was editing in LibreOffice. I'll copy it to the test vm again and see what happens. |
| 10:39 |
jeff |
oh, if you're using Spreadsheet::Read and the input barcode file is different between test and production, I suspect the input file is your issue now. |
| 10:40 |
* jeff |
nods |
| 10:42 |
Dyrcona |
yeahp. It's chugging away, now. |
| 10:42 |
Dyrcona |
GIGO |
| 10:44 |
Dyrcona |
So, they want the already deleted copies checked in, so I should go back to the cstore session or editor, remove the deleted=>'f', etc., etc., and so on and so forth. |
| 10:44 |
Dyrcona |
jeff++ |
| 10:44 |
Dyrcona |
csharp++ |
| 10:54 |
Dyrcona |
OK. It is working now. I forgot to change to check in by copy id, though. I'm getting asset copy not found on the deleted ones. |
| 12:35 |
|
collum joined #evergreen |
| 13:06 |
|
Dyrcona joined #evergreen |
| 14:26 |
|
rjackson_isl_hom joined #evergreen |
| 15:19 |
|
alynn26_away joined #evergreen |
| 15:38 |
|
bott_grpl joined #evergreen |
| 15:48 |
bott_grpl |
Can someone point me toward where the summary.bibCallNumber is populated in the bib summary for /eg2/en-US/staff/catalog/record/ ? It's showing me LC info, where I've got Dewey showing in the volcopy editor, which is what we want. |
| 16:07 |
berick |
bott_grpl: eg2/src/app/share/catalog/bib-record.service.ts |
| 16:07 |
berick |
it calls open-ils.cat.biblio.record.marc_cn.retrieve |
| 16:10 |
jeffdavis |
... which in turn will pull the call number from the MARC record; looks like you may be able to set cat.default_classification_scheme to Dewey to make it pull a Dewey call number instead of LC |
| 16:11 |
bott_grpl |
The scheme is set, which was part of my confusion. |
| 16:13 |
bott_grpl |
As it works as expected in the volcopy editor, but not the record display in /eg2/ |
| 16:16 |
jeffdavis |
It should be grabbing the first value it finds from the list of fields specified in the field column of asset.call_number_class for your default classification scheme |
| 16:17 |
jeffdavis |
at least, that's my read of the source for the open-ils.cat.biblio.record.marc_cn.retrieve call |
| 16:18 |
berick |
srfsh# request open-ils.cat open-ils.cat.biblio.record.marc_cn.retrieve 1 |
| 16:18 |
berick |
change record ID to taste |
| 16:19 |
bott_grpl |
Ooo, a Status: 500 from that srfsh request is probably bad! |
| 16:21 |
bott_grpl |
Ugh. id typo |
| 16:22 |
bott_grpl |
But exactly what I was looking for! |
| 16:22 |
bott_grpl |
berick++ jeffdavis++ |
| 17:31 |
|
jvwoolf left #evergreen |
| 18:00 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 18:45 |
pinesol |
[evergreen|Jason Stephenson] Lp 1862652: pingest.pl reingest record attributes fix - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=5ac76ef> |
| 23:03 |
|
alynn26_away joined #evergreen |
