| Time |
Nick |
Message |
| 00:13 |
pinesol_green |
[evergreen|Jane Sandberg] Docs: adding backup information to cli manual - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=7ecad6e> |
| 02:27 |
|
kipd joined #evergreen |
| 06:01 |
pinesol_green |
News from qatests: Test Success <http://testing.evergreen-ils.org/~live> |
| 07:18 |
|
rjackson_isl joined #evergreen |
| 08:15 |
|
collum joined #evergreen |
| 08:43 |
|
mmorgan joined #evergreen |
| 09:00 |
|
JBoyer joined #evergreen |
| 09:00 |
JBoyer |
miker, you around? |
| 09:01 |
JBoyer |
Or, anyone else with some insider knowledge re: 3.0 search. |
| 09:01 |
JBoyer |
? |
| 09:01 |
|
Dyrcona joined #evergreen |
| 09:05 |
* csharp |
is curious about JBoyer's search concern since we're testing 3.0 right now |
| 09:06 |
JBoyer |
I've found a difference in the queries where staff can't accurately limit to a single location but patrons can. I was hoping to get some background on why. |
| 09:06 |
JBoyer |
Also, has your 3.0 testing turned up anything similar? (or, rather, have you checked?) |
| 09:12 |
csharp |
not that I know of - I can check |
| 09:14 |
mmorgan |
JBoyer: I just did a quick search logged in as a staff user on a 3.0 system, and am also seeing a problem with limiting. |
| 09:15 |
JBoyer |
Ok. So good news, I'm neither crazy* nor running a damaged system. (*At least not for this reason) |
| 09:15 |
mmorgan |
:) |
| 09:15 |
Dyrcona |
:) |
| 09:16 |
mmorgan |
JBoyer: Is this the web client, xul client, or both? |
| 09:17 |
JBoyer |
I'm pulling some queries out to do a proper diff, but I easily see by eye 2 spots that are different staff vs patrons related to the search.calculate_visibility_attribute_test lines. |
| 09:17 |
JBoyer |
mmorgan, both. |
| 09:20 |
csharp |
yep, I can confirm - regular non-logged in OPAC search limits fine, same search in web client brings back items not owned by selected location |
| 09:53 |
|
jvwoolf joined #evergreen |
| 10:02 |
|
mmorgan1 joined #evergreen |
| 10:02 |
|
rlefaive_ joined #evergreen |
| 10:24 |
|
collum_ joined #evergreen |
| 10:57 |
JBoyer |
csharp, mmorgan1, if you would like to "This affects me too!" or confirm bug 1723977 I would appreciate it |
| 10:57 |
pinesol_green |
Launchpad bug 1723977 in Evergreen "Searching specific location in Advanced Search not limiting correctly in 3.0" [Undecided,New] https://launchpad.net/bugs/1723977 |
| 10:58 |
JBoyer |
Now to see if it's a fool's errand to try to visit an ikea that's only been open a week... |
| 11:08 |
|
kmlussier joined #evergreen |
| 11:08 |
rhamby |
If we never see JBoyer again now we will know why.... |
| 11:08 |
berick |
rhamby: he's living in 450 sq feet now |
| 11:09 |
rhamby |
heh |
| 11:13 |
|
Christineb joined #evergreen |
| 11:13 |
miker |
hrm... well, I'm around now |
| 11:15 |
miker |
re the search stuff above, I looked at this last week with an upgraded system and didn't see the behavior. which obv doesn't mean it's not happening, just that something's different |
| 11:19 |
|
mmorgan joined #evergreen |
| 11:19 |
kmlussier |
I see it too on one of my test VMs. |
| 11:30 |
|
rlefaive joined #evergreen |
| 11:46 |
kmlussier |
Never mind. I don't see it on my VM. I was just seeing the standard gray bibs that always come up in a staff search. |
| 11:46 |
* kmlussier |
drinks more coffee. |
| 11:47 |
kmlussier |
This dataset shouldn't have any copy-less bibs, but I guess that's another problem to solve in my spare time on a later date. |
| 12:02 |
|
khuckins joined #evergreen |
| 12:58 |
|
yboston joined #evergreen |
| 12:59 |
|
khuckins joined #evergreen |
| 13:08 |
|
roycroft joined #evergreen |
| 13:27 |
|
JBoyer joined #evergreen |
| 13:27 |
JBoyer |
ikea update: I'm an idiot. Never do that. |
| 13:27 |
Dyrcona |
Ha! |
| 13:27 |
Dyrcona |
JBoyer: We were taking bets on whether or not you'd return. |
| 13:28 |
* Dyrcona |
is going to apply patches to Lp 1527713 in production this week. |
| 13:28 |
pinesol_green |
Launchpad bug 1527713 in Sahara "[DOC] DevStack git repo link is incorrect in Sahara Dev Ref" [Low,Fix released] https://launchpad.net/bugs/1527713 - Assigned to Akanksha Agrawal (akanksha-aha) |
| 13:28 |
Dyrcona |
No, not that one.... |
| 13:29 |
Dyrcona |
Let's try again: Lp 1527731 |
| 13:29 |
pinesol_green |
Launchpad bug 1527731 in OpenSRF "Query for OPAC copies can be extremely slow" [Undecided,New] https://launchpad.net/bugs/1527731 |
| 13:29 |
JBoyer |
Didn't even leave the car. Took 2-3 mins to traverse the parking lot straight to the exit. Found a VCR at a nearby goodwill instead. (my shopping list is rather scattered) |
| 13:29 |
* Dyrcona |
thinks those patches may fix other issues. |
| 13:36 |
Dyrcona |
miker: After installing the patches for Lp 1527731, should I restart memcached and/or clear out the template caches in /tmp? |
| 13:36 |
JBoyer |
Not knowing that much of the history of perl hashes or future plans, I worry that it may hide several issues that will all hit us at once if the ability to specify hash seeds ever goes away. |
| 13:36 |
pinesol_green |
Launchpad bug 1527731 in OpenSRF "Query for OPAC copies can be extremely slow" [Undecided,New] https://launchpad.net/bugs/1527731 |
| 13:37 |
Dyrcona |
JBoyer: I hear that a new solution is going to be on offer that does not rely on Perl's behavior. |
| 13:38 |
Dyrcona |
My goal is to solve what seems like an immediate problem while that other solution is developed. |
| 13:39 |
JBoyer |
Makes sense to ne, |
| 13:39 |
JBoyer |
me, too. |
| 13:39 |
Dyrcona |
:) |
| 13:39 |
Dyrcona |
I think it makes sense to restart memcached, because it seems like caches misses are the issue. |
| 13:40 |
Dyrcona |
But, doing so in the middle of the day would kick people out who are already logged in, and since no one has reported issues with sessions timing out.....Maybe I should just leave memcached alone? |
| 13:42 |
JBoyer |
Slow cache key attrition vs sudden angry key-table flipping, seems easy to me. ;) |
| 13:42 |
Dyrcona |
Yeah, when you put it that way..... |
| 13:43 |
Dyrcona |
:) |
| 13:48 |
|
rjackson_isl_ joined #evergreen |
| 14:05 |
|
rlefaive joined #evergreen |
| 14:10 |
miker |
Dyrcona: so, there looks like there could be a bug (fixed by those hash key order related patches) in open-ils.fielder.flattened_search.prepare, but otherwise, memcache keys don't depend on hash key order at all. |
| 14:11 |
Dyrcona |
miker: OK. Thanks. I'm still going to try this because I'm getting all kinds of crazy bug reports since the upgrade to 2.12, and I want to eliminate as much as possible before blaming it all on chunking in OpenSRF 2.5.2. :) |
| 14:13 |
miker |
sure thing. all data points appreciated |
| 14:17 |
dbs |
we haven't had too many crazy bug reports since upgrading to 2.12 and addressing the chunking issues and a few nginx issues |
| 14:19 |
Dyrcona |
dbs: Do you have any patches/configuration beyond just installing OpenSRF 2.5.2? |
| 14:24 |
dbs |
we still increased ejabberd's max message length to 2000000 or whatever |
| 14:25 |
dbs |
sorry, max_stanza_size is now 100000 |
| 14:25 |
|
jvwoolf joined #evergreen |
| 14:26 |
dbs |
We have lots of records with UTF8 chars so trying to be careful |
| 14:26 |
Dyrcona |
OK. I might try that. I'm searching logs now for relevant messages. |
| 14:27 |
Dyrcona |
I'm getting a lot of "not connected to the network" errors, and I'm trying to find a cause. |
| 14:27 |
Dyrcona |
It's easy, either. It's not like it's 1 brick all the time. It's any of them at any time. |
| 14:28 |
dbs |
I recently found our open-ils.search parent process died and left a child behind that made every search result be zero hits |
| 14:28 |
dbs |
Not sure what prompted that but restarting open-ils.search fixed it (didn't want to spend the time on forensics) |
| 14:29 |
Dyrcona |
The most fun that I've had was when opensrf.settings died on one of the bricks, and pretty much everything that tried to talk to it up and died, too. |
| 14:31 |
dbs |
oh yeah that's never a good situation |
| 14:32 |
dbs |
noticed today that I was asking marc_export to encode stuff in "UTF8" and the check for the encoding output for binmode was hardcoded to look for "UTF-8", I might throw together a patch to regex it for /UTF-?8/ |
| 14:32 |
dbs |
wide_characters-- |
| 14:33 |
Dyrcona |
Yeah, looks like I'll need to increase max_stanza_size after all. "Text of error message received from Jabber: XML stanza is too big" |
| 14:35 |
Dyrcona |
Twice before the upgrade and 49 times since. |
| 15:22 |
Dyrcona |
Yeah, so I think my issues may be more likely caused by max_stanza_size still being too low with bundling and chunking. |
| 15:24 |
pinesol_green |
[evergreen|Cesar Velez] LP#1719967 - Add alert message field to volcopy editor - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=ca88dba> |
| 15:30 |
pinesol_green |
[evergreen|Cesar Velez] LP#1716962 - honor In-House Use: num of uses threshold setting - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=3175bcc> |
| 15:34 |
csharp |
dbs: I woke up to a dead open-ils.search parent with orphans last week too |
| 15:34 |
csharp |
same deal - just restarted and kicked the forensics can down the road for the next time |
| 15:36 |
csharp |
Dyrcona: I've had 751 NOT CONNECTED TO THE NETWORKs today so far |
| 15:36 |
csharp |
all but one are open-ils.search |
| 15:38 |
pinesol_green |
[evergreen|Bill Erickson] LP#1717010 Webstaff print/export full grid - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=23ab19f> |
| 15:38 |
Dyrcona |
I've had 42 today if the sylog server can be believed. |
| 15:39 |
Dyrcona |
They've all been clients on the private router. |
| 15:40 |
pinesol_green |
[evergreen|Jason Boyer] LP1717025: Persist Specific Due Dates Until Logout - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=87dc85c> |
| 15:40 |
pinesol_green |
[evergreen|Cesar Velez] LP#1717025 - make Specific Due Dates Until Logout in Patron Checkout - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=5e479ec> |
| 15:41 |
mmorgan |
We've had 169 NOT CONNECTED.. today :-( |
| 15:41 |
mmorgan |
all open-ils.search |
| 15:56 |
JBoyer |
csharp, Dyrcona, mmorgan: what finally took care of the open-ils.search deaths was this: https://bugs.launchpad.net/opensrf/+bug/1697029/comments/10 |
| 15:56 |
pinesol_green |
Launchpad bug 1697029 in OpenSRF "Service Listeners Crash when Using an Undefined Value for Syscalls" [High,Fix released] |
| 16:00 |
JBoyer |
The cause of the issue here seemed to be a delay in auth-ing new jabber connections. That patch tries to prevent crashes from writing to dead pids, but it can't affect the cause of the dead pids. |
| 16:03 |
|
khuckins joined #evergreen |
| 16:11 |
mmorgan |
JBoyer: We didn't implement those tweaks, but are running OpenSRF 2.5.2. The search process no longer crashes, but we still see those dead pids. |
| 16:13 |
JBoyer |
The patches on that bug just keep the listener from crashing, those tweaks prevent the children from crashing (as often, I suspect it's still possible) |
| 16:13 |
csharp |
looks like a lot of ours are bug 1514549 continuing to pester us |
| 16:13 |
pinesol_green |
Launchpad bug 1514549 in Evergreen "Pub date sorting/filtering causes slow OPAC search" [Undecided,New] https://launchpad.net/bugs/1514549 |
| 16:14 |
csharp |
the 2 I've dug into at length so far are 53s+ queries |
| 16:14 |
csharp |
basically people trying to see the newest books |
| 16:16 |
csharp |
hmm - is there a good way to track down the source (IP) for a bib search? |
| 16:17 |
csharp |
a lot of these appear to be literally the same search |
| 16:17 |
csharp |
I'm thinking it's something automated or pre-set in an OPAC or library web page |
| 16:18 |
csharp |
-- bib search: #CD_documentLength #CD_meanHarmonic #CD_uniqueWords core_limit(10000) limit(1000) badge_orgs(1) estimation_strategy(inclusion) item_type(a,t;query=the edge of nowhere;qtype=title;locg=124;query= George Elizabeth 1949 ;qtype=author;facet=subject|name[Vance, Jack 1916-2013];facet=author|corporate[North Carolina Dept. of Archives and History.];qtype=author;query=Orton H |
| 16:19 |
csharp |
F;long_facet=subjecttopic;query=Elmer Lucius Q C Lucius Quintius Cincinnatus 1793 1883;qtype=author;facet=subject|topic[History];qtype=author;query=Porter Bill 1939 2014;page=6;long_facet=subjectgeographic) depth(0) |
| 16:19 |
csharp |
"Cincinnatus" appears 771 times this hour so far, so um... |
| 16:21 |
* Dyrcona |
only has 15 dead open-ils.search drones and/or listeners since the upgrade. |
| 16:21 |
Dyrcona |
Someone has a thing for Roman generals..... |
| 16:22 |
* csharp |
is completely at a loss |
| 16:22 |
miker |
csharp: that's a broken query altogether. like, a broken URL |
| 16:23 |
miker |
somehow the one param swallowed everything the followed it in the url, IOW |
| 16:23 |
miker |
s/the/ |
| 16:24 |
miker |
which suggests a url encoding issue that confused CGI.pm |
| 16:25 |
* csharp |
tries to think of possibilities that would explain this happening at such a crazy frequency |
| 16:25 |
miker |
IIRC, there used to be ways to make the adv search page do that |
| 16:25 |
miker |
a mix of param separators, maybe? |
| 16:27 |
miker |
well, if you forget the embedded query and just consider '#CD_documentLength #CD_meanHarmonic #CD_uniqueWords core_limit(10000) limit(1000) badge_orgs(1) estimation_strategy(inclusion) item_type(a,t) depth(0) "Cincinnatus"' then it looks more sane... :) |
| 16:27 |
* mmorgan |
has a recollection of a recent irc discussion where there was electronic signage continually hitting the catalog. Can't find it in the logs, though. |
| 16:27 |
miker |
mmorgan: ew! |
| 16:27 |
JBoyer |
csharp, as for finding the IP, grep the gateway for parts of that to get something similar to an Apache logline. |
| 16:28 |
miker |
and, also, I have one instance that is constantly and consistently pounded by "show me all things at library XXX", for various values of XXX, and it pages through those 1000 (a superpage) at a time, forever |
| 16:28 |
miker |
which sure looks like something harvesting to me |
| 16:30 |
miker |
but it's claimed no such is known to be happening, or known to be allowed, by the IP owners |
| 16:30 |
miker |
csharp: so, I feel your pain :) |
| 16:31 |
miker |
and, yeah, gateway and apache logs will help you with the IP (if you're not using nginx or pound) |
| 16:34 |
csharp |
we are using nginx, but I'm still seeing IPs so far |
| 16:34 |
csharp |
"hey, there's Unique!" |
| 16:36 |
* csharp |
has no problem pulling out the iptables drop guns if necessary |
| 16:37 |
miker |
"you can have your ILS back when you get the cat off the keyboard!" |
| 16:37 |
csharp |
given that there were 465 instances of "Cincinnatus" in the logs from midnight, I'm thinking this is not a library (or cat) |
| 16:39 |
csharp |
bleh - yeah nginx is masking this |
| 16:41 |
csharp |
ah nginx log reveals a bot |
| 16:43 |
csharp |
https://www.semrush.com/bot/ |
| 17:00 |
csharp |
wow - blocking that cleared up the logs considerably ;-) |
| 17:05 |
csharp |
... which allows me to see a lot of Apache2::RequestIO::print: (32) Broken pipe at /usr/lib/perl5/Template.pm line 180, |
| 17:06 |
|
mmorgan left #evergreen |
| 17:06 |
miker |
csharp: I find that often means (the equiv of) someone hit the "stop" button in the browser. fwiw. |
| 17:06 |
miker |
if it's a bot, it means it dropped the socket on the floor after some timeout |
| 17:07 |
miker |
could be a user clicking a link again, or another link, after waiting |
| 17:07 |
miker |
there's a patch for that :) ... maybe in 3.0? |
| 17:08 |
|
jvwoolf left #evergreen |
| 17:11 |
csharp |
miker: good to know |
| 17:44 |
Bmagic |
Maybe someone has had this issue before and you just know the answer - [% circ.target_copy.call_number.record.simple_record.title %] in a action trigger checkout.due notification |
| 17:45 |
Bmagic |
It is hit and miss. The title shows up sometimes and sometimes it doesn't for exact same email sometimes. |
| 17:45 |
Bmagic |
An example might be 2 items in the same email. The first loop doesn't show the title, and the second one does |
| 17:46 |
Bmagic |
None of the titles that should appear have diacritics or any other artifactsthat would lead me to believe it was an issue with the title itself or parsing it thereof. |
| 17:48 |
csharp |
maybe the wrong kind of join in the fieldmapper linkage? (but that probably would result in perl-level errors rather than blank data) |
| 17:54 |
berick |
Bmagic: you've confirmed the bibs in question have reporter.simple_record entries? |
| 17:54 |
berick |
... with titles |
| 18:01 |
pinesol_green |
News from qatests: Test Success <http://testing.evergreen-ils.org/~live> |
| 19:04 |
rhamby |
fyi, removing roles of some wordpress accounts for those that haven't been active in 8+ years |
| 19:12 |
csharp |
wow - that bot really was wreaking havoc for us - error/warn logs are very reasonable now |
| 19:12 |
csharp |
@band add Wreaking Havoc |
| 19:12 |
pinesol_green |
csharp: Band 'Wreaking Havoc' added to list |
| 20:13 |
csharp |
@blame web crawlers |
| 20:13 |
pinesol_green |
csharp: web crawlers stole csharp's ice cream! |
| 20:13 |
csharp |
@dunno |
| 20:13 |
pinesol_green |
csharp: It reads like a Nigerian 419 scam, but I think it is a sincere question sent to the wrong list. |
| 20:13 |
csharp |
that should've been a @quote |
| 20:14 |
csharp |
@quote random |
| 20:14 |
pinesol_green |
csharp: Quote #43: "commit 4755baac: There's tears in my coffee. These are not tears of joy." (added by gmcharlt at 05:36 PM, January 25, 2013) |
| 20:14 |
csharp |
4755baac |
| 20:14 |
pinesol_green |
csharp: [evergreen|dbs] And... keep the index creation on asset.call_number in the right place - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=4755baa> |
| 20:53 |
|
kiswe joined #evergreen |
| 20:53 |
|
kiswe left #evergreen |
| 22:26 |
|
eady joined #evergreen |
| 22:33 |
|
dbwells joined #evergreen |
| 23:11 |
|
dbwells_ joined #evergreen |
