| Time |
Nick |
Message |
| 03:19 |
|
dbwells joined #evergreen |
| 07:10 |
|
rjackson_isl_hom joined #evergreen |
| 07:18 |
csharp |
@quote add <Bmagic> It's as if millions of bugs all cried out at once, and were silenced. |
| 07:18 |
pinesol |
csharp: The operation succeeded. Quote #213 added. |
| 08:16 |
|
mantis1 joined #evergreen |
| 08:21 |
|
stephengwills joined #evergreen |
| 08:35 |
|
rfrasur joined #evergreen |
| 08:44 |
|
mmorgan joined #evergreen |
| 09:03 |
|
Dyrcona joined #evergreen |
| 09:04 |
|
dbwells joined #evergreen |
| 09:07 |
Bmagic |
csharp++ # Haha... let me see if I can use the commands to search the quotes for the other ones I authored |
| 09:16 |
Bmagic |
@quotes Bmagic |
| 09:16 |
pinesol |
Bmagic: Have you confirmed your ISBN SPIDs with your service provider? |
| 09:17 |
Bmagic |
@quote Bmagic |
| 09:17 |
pinesol |
Bmagic: Down time is a fact of business when you're a poor 501c3 corporation. |
| 09:17 |
Bmagic |
struck out |
| 09:18 |
Bmagic |
Pretty sure this is a feature? Though it's not documented here https://wiki.evergreen-ils.org/doku.php?id=community:using-pinesol_green |
| 09:18 |
Bmagic |
where's the source code for pinesol? |
| 09:19 |
Bmagic |
@quote search Bmagic |
| 09:19 |
pinesol |
Bmagic: 4 found: #149: "< Bmagic> my favorite viruses are the ones...", #172: "< Bmagic> I like to keep my systems at a power...", #179: "<Bmagic> It's amazing how often 30 days comes...", and #213: "<Bmagic> It's as if millions of bugs all cried..." |
| 09:19 |
Bmagic |
bingo |
| 09:19 |
Bmagic |
@quote #149 |
| 09:19 |
pinesol |
Bmagic: http://www.firstpersontetris.com/ |
| 09:20 |
Bmagic |
@quote 149 |
| 09:20 |
pinesol |
Bmagic: Your computer account is overdrawn. Please reauthorize. |
| 09:20 |
Bmagic |
@quote show #149 |
| 09:20 |
pinesol |
Bmagic: NOBODY EXPECTS THE SPANISH INQUISITION! |
| 09:20 |
Bmagic |
man I suck at this |
| 09:20 |
Bmagic |
@quote help |
| 09:20 |
pinesol |
Bmagic: https://i.imgur.com/m8EySrW.gifv |
| 09:21 |
Bmagic |
I clicked on that thinking it would help me. I was wrong |
| 09:22 |
Bmagic |
@help quote |
| 09:22 |
pinesol |
Bmagic: Error: There is no command "quote". However, "Quote" is the name of a loaded plugin, and you may be able to find its provided commands using 'list Quote'. |
| 09:22 |
Bmagic |
@help Quote |
| 09:22 |
pinesol |
Bmagic: Error: There is no command "quote". However, "Quote" is the name of a loaded plugin, and you may be able to find its provided commands using 'list Quote'. |
| 09:22 |
Bmagic |
@list Quote |
| 09:22 |
pinesol |
Bmagic: add, change, get, random, remove, replace, search, and stats |
| 09:23 |
Bmagic |
@quote get #149 |
| 09:23 |
pinesol |
Bmagic: (quote get [<channel>] <id>) -- Gets the quote with id <id> from the quote database for <channel>. <channel> is only necessary if the message isn't sent in the channel itself. |
| 09:23 |
Bmagic |
@quote get 149 |
| 09:23 |
pinesol |
Bmagic: Quote #149: "< Bmagic> my favorite viruses are the ones claiming to be the cure" (added by csharp at 10:32 AM, March 22, 2016) |
| 09:24 |
Bmagic |
let this be a lesson to everyone! - and now, I'm editing the wiki |
| 09:25 |
Dyrcona |
Bmagic: You can do all of that in a private conversation with pinesol so as not to spam the channel. |
| 09:25 |
Bmagic |
dang it |
| 09:25 |
Dyrcona |
@quote get 149 |
| 09:25 |
pinesol |
Dyrcona: Quote #149: "< Bmagic> my favorite viruses are the ones claiming to be the cure" (added by csharp at 10:32 AM, March 22, 2016) |
| 09:25 |
Bmagic |
sorry. I'll be sure and mention that in my wiki edit |
| 09:27 |
Dyrcona |
You need to add the channel to the command when doing a private chat because the bot keeps track of quotes, etc. by channel. |
| 09:27 |
Dyrcona |
@dunno |
| 09:27 |
pinesol |
Dyrcona: Have you tried throwing it across the room? |
| 09:27 |
Bmagic |
yep - the help message above mentioned that |
| 09:47 |
csharp |
I'm mostly interested to see all the @dunno entries I'd forgotten about :-) |
| 09:51 |
Dyrcona |
So, I have to figure out why action.emergency_closing_stage_2_hold is so slow. I cancelled two yesterday that ran for a day, and 1 this morning. |
| 09:51 |
Dyrcona |
I'm gonna open a local ticket so I don't forget about it. |
| 10:09 |
|
dbwells joined #evergreen |
| 10:14 |
Dyrcona |
Does anyone else think that the emergency closing code is too slow? I'm asking because maybe it's just us. |
| 10:19 |
pinesol |
[evergreen|Garry Collum] LP1868147-Titles in carousel run together in mobile display. - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=ef55734> |
| 10:20 |
|
dbwells joined #evergreen |
| 10:23 |
|
alynn26 joined #evergreen |
| 10:32 |
|
dbwells joined #evergreen |
| 10:43 |
csharp |
Dyrcona: that sounds familiar, but I never dug in |
| 11:06 |
|
dbwells joined #evergreen |
| 11:06 |
|
dbwells_ joined #evergreen |
| 11:11 |
Dyrcona |
So, search is busted on my training server after an upgrade from 3.2.10 to 3.5.2. I've tried a reingest (with pingest.pl), checked that the opac templates (they look OK), checked the logs for errors (nothing looks related to search), and I've verified that we have all of the upgrades from upgrade directory installed. |
| 11:11 |
Dyrcona |
Anyone got any ideas of what to try next? |
| 11:15 |
csharp |
check activity logs for the searches and trace them through the osrfsys log? |
| 11:15 |
csharp |
(assuming those are broken out) |
| 11:21 |
Dyrcona |
Well, I have more information since I typed that. Others have updated our internal ticket. |
| 11:22 |
Dyrcona |
Looks like search is working in the "experimental catalog." If you do a search that returns a single hit, it works. |
| 11:22 |
Dyrcona |
What we get in the results is a list of "blank" results, so it is looking like a possible template issue. |
| 11:24 |
Dyrcona |
I have removed our local templates and checked out those from origin/tags/rel_3_5_2 and those needed for curbside and stripe backports. I'll restore our templates one at a time until search breaks, assuming it's working, now. |
| 11:26 |
Dyrcona |
Well, nope. That didn't fix it. |
| 11:28 |
Dyrcona |
Is anyone running 3.5.2 in production? |
| 11:35 |
Dyrcona |
Here's a screen shot with our customizations removed: https://drive.google.com/file/d/1MINHLMKnaPKx10KZ98RgyA6v0ay8OW7W/view?usp=sharing |
| 11:39 |
Dyrcona |
And, here's one with our customizations: https://drive.google.com/file/d/1_2o2k7wy-t5WJnxrGg93OEmY_VvR_WLW/view?usp=sharing |
| 11:51 |
Dyrcona |
So, I'm going to do a git clean -x -f -d and rm -rf /openils/var/templates, and then do a clean install. IIRC, I did rm -rf /openils before installing everything fresh. I suppose I could also try deleting /usr/local/share/perl/5.26.1/OpenILS. |
| 12:00 |
Dyrcona |
At 11:22:10 when I said search was working in the experimental catalog, I meant that it was getting multiple results. The single hit statement was about the TTOPAC. Also, the experimental catalog information is anecdotal. I haven't checked it, myself. |
| 12:00 |
Dyrcona |
And, it's lunch time. |
| 12:09 |
|
jihpringle joined #evergreen |
| 12:13 |
Bmagic |
Dyrcona: making the templates Evergreen stock is a good first step I would say. Next, I would reinstall OpenSRF/Evergreen. Nothing in the logs to go on? |
| 12:13 |
Bmagic |
Dyrcona: yes, the emergency closing code is slow. I've seen it run for days sometimes |
| 12:18 |
|
Cocopuff2018 joined #evergreen |
| 12:27 |
|
sandbergja_ joined #evergreen |
| 12:35 |
jeffdavis |
We're somewhere between 3.5.1 and 3.5.2 in production if it helps. |
| 12:53 |
|
jihpringle joined #evergreen |
| 12:59 |
Dyrcona |
Bmagic: I already reverted to stock templates and it was the same I shared two screen shots, in case anyone missed them. |
| 13:00 |
Bmagic |
I saw them, I was agreeing with your path |
| 13:00 |
Bmagic |
check postgres logs - might be a missing function |
| 13:09 |
|
alynn26_away joined #evergreen |
| 13:11 |
JBoyer |
Dyrcona, this may or may not help, but maybe make sure fm_IDL.xml was actually updated? I remember seeing an issue like yours a couple years ago but don't recall the cause. |
| 13:11 |
JBoyer |
I don't know how that file could be out of date but I've seen it happen in the past. |
| 13:21 |
Dyrcona |
Despite my best efforts at comparing the upgrade log with the upgrade scripts in the Open-ILS/src/sql/Pg/upgrade directory, it looks like some upgrades are missing: https://drive.google.com/file/d/1QMg6vIK7dAC2E-_epc4jVoN4zzPX9P_G/view?usp=sharing |
| 13:22 |
Dyrcona |
I wonder if some of the 3.3. or 3.4 upgrades failed to make it into 3.5? |
| 13:25 |
Bmagic |
Dyrcona: I think you're getting close! |
| 13:31 |
Dyrcona |
I'm not so sure because the missing numbers are all over 1200, but I'll check them against 3.3 and 3.4 anyway. |
| 13:33 |
Dyrcona |
None of the missing upgrades appear in rel_3_3 nor in rel_3_4. |
| 13:36 |
Dyrcona |
So maybe code made it into 3.5 but the db upgrades were missed? However, jeffdavis says they're on 3.5, so that's not likely to be the problem. |
| 13:38 |
|
collum joined #evergreen |
| 13:41 |
Dyrcona |
All right, so it looks like the missing relations are local additions, and none of them have anything to do with search. |
| 13:42 |
Dyrcona |
The two errors about composite types are the only ones that look relevant. |
| 13:43 |
Dyrcona |
WTF? SELECT * FROM search.search_result AS x WHERE 1=0; |
| 13:48 |
Dyrcona |
Wait a minute. The queries for the missing tables are the same. Is this related to fieldmapper? Guess, I'll listen to JBoyer and see if the IDL isn't hosed. |
| 13:52 |
Dyrcona |
xmllint reports no errors. |
| 13:54 |
|
sandbergja_ joined #evergreen |
| 14:03 |
Dyrcona |
Oddly enough, I can't find the missing relations in our production databases, either, but they don't spit out those errors. |
| 14:07 |
Dyrcona |
The code producing SELECT * FROM %s AS x WHERE 1=0; has been there since 2010. |
| 14:10 |
Dyrcona |
"Curiouser and curiouser," said Alice. |
| 14:12 |
Dyrcona |
Now, I can't find those tables in our IDL, but I swear they were there before... |
| 14:12 |
csharp |
@who moved Dyrcona's tables while listening to the new album from [band]? |
| 14:12 |
pinesol |
awitter moved Dyrcona's tables while listening to the new album from Sentient Peach. |
| 14:13 |
Dyrcona |
Heh! |
| 14:14 |
csharp |
@praise [band] |
| 14:14 |
* pinesol |
Huge Bean Weevil is the very model of a modern major hacker |
| 14:14 |
Dyrcona |
Turns out I accidentally included an extra character when searching via the editor. |
| 14:18 |
Dyrcona |
diffing our custom IDL with the one from rel_3_5 doesn't turn up anything striking. We added the acsp object for curbside, we have an extra link in 1 object (that should be added to Lp, btw), an extra blank and our custom sources at the "end." |
| 14:18 |
Dyrcona |
Both IDLs come out as valid XML when I check them. |
| 14:25 |
Dyrcona |
The composite types mentioned in the pgerrors.txt have been in the IDL since 2009. |
| 14:25 |
Dyrcona |
I'm stumped. |
| 14:30 |
jeffdavis |
Dyrcona: there are a couple of test scripts for search in Open-ILS/src/support-scripts/test-scripts/ - do they show anything interesting? |
| 14:30 |
jeffdavis |
my next guess would be an issue with visibility attrs |
| 14:37 |
Dyrcona |
jeffdavis: I haven't run them. |
| 14:39 |
Dyrcona |
I don't think the problem is really search or the database, because it works with the experimental Angular catalog, but I'll give those scripts a shot. I still think it has to be the templates or something in the Perl. |
| 14:57 |
Dyrcona |
So, query_tests.pl returns results on some of the searches. Naturally, many of them return nothing because it looks like it is written for the concerto dataset. |
| 15:02 |
Dyrcona |
The perl modules all check out. Perl -c reports "syntax OK" on all of them. |
| 15:02 |
Dyrcona |
Doesn't mean that there aren't bugs. |
| 15:07 |
csharp |
"your horrible code has the proper syntax" |
| 15:09 |
Dyrcona |
If Perl can be said to have syntax. :P |
| 15:10 |
csharp |
TMTOWTDI can definitely be a weakness |
| 15:29 |
Bmagic |
lol |
| 15:30 |
Dyrcona |
TimToady is a registered nick. :) |
| 15:31 |
Dyrcona |
So, how do I use the alternate catalog/eg2 on 3.5? |
| 15:31 |
|
khuckins_ joined #evergreen |
| 15:31 |
Dyrcona |
I logged in at eg2/en-US/staff but got redirected to the AngularJS staff client. |
| 15:33 |
|
malexander joined #evergreen |
| 15:33 |
JBoyer |
There may be an OUS gating it from appearing in the Cataloging menu, not sure. |
| 15:34 |
JBoyer |
Unless that was 3.4... |
| 15:34 |
Dyrcona |
Hey! Waitaminute! After doing that, I'm getting search results in the staff client, but still none in the regular OPAC. |
| 15:34 |
Dyrcona |
Maybe that's what John meant in the ticket and I misunderstood? |
| 15:34 |
Dyrcona |
Still doesn't tell me much. |
| 15:36 |
Dyrcona |
Ah, wait. Now, when I search for the whole consoritum, with 10 results, and not just my branch, with 2, I get blank results.... |
| 15:40 |
Dyrcona |
It's working that way in the training OPAC, after logging out, closing the window, etc. |
| 15:41 |
Dyrcona |
So, it could be related to db performance or something. Think I'll shut it all down, install updates, reboot and vaccum the db. |
| 15:41 |
|
mantis1 left #evergreen |
| 16:00 |
|
jvwoolf joined #evergreen |
| 16:02 |
Dyrcona |
Well, isn't that nice... => /var/lib/postgresql is using 86.5% of 511.73GB |
| 16:04 |
Dyrcona |
Turns out some of those missing relations are not custom tables/views. I'll have to look into that tomorrow. |
| 16:05 |
csharp |
could be holding onto xlog for some reason? unless it's legit that big |
| 16:25 |
Dyrcona |
Our production database is 390GB after a full vacuum. |
| 16:26 |
Dyrcona |
This one is a copy with a couple of tables truncated. |
| 16:27 |
Dyrcona |
I don't think autovacuum helps us all that much, tbh. The production database is 413GB at the moment. I'm pretty sure that most of the extra 20GB is dead space. |
| 16:38 |
Dyrcona |
Last time that I full vacuumed production, df indicate it was using something like 796GB and afterward is was 390GB. |
| 16:46 |
csharp |
Dyrcona: yeah - I've not fully figured out the art of autovacuum - it's a trade off between performance and... well, performance, I guess |
| 16:47 |
Dyrcona |
We do a vaccum analyze in production every Saturday morning. |
| 16:52 |
csharp |
Sunday night for us |
| 16:54 |
csharp |
okay - I'm back to having to restart opensrf every hour or so to prevent actor drone exhaustion - trying to remember where we left things... plan was to push your diagnostic fix so we can see in the browser which UIs are behaving badly, right |
| 16:55 |
csharp |
but the only thing we saw flat-out broken was dojo stuff, which were considering irrelevant to the fix, but Acq users are impacted |
| 16:55 |
csharp |
so the choice is acq PO loading versus drone saturation right now |
| 16:56 |
* csharp |
meant to tag berick somewhere in there :-) |
| 16:57 |
|
sandbergja_ joined #evergreen |
| 16:58 |
Dyrcona |
:) |
| 17:02 |
csharp |
gmcharlt: (or someone EOLI-ish), just FYI, looks like certbot needs a kick on yeti.esilibrary.com - cert expired on 9/29/2020 |
| 17:03 |
csharp |
I was curious about the Angular acq stuff (which I should already know more about, but don't ;-)) |
| 17:14 |
jeffdavis |
csharp: can you tell where the drone saturation is being triggered? |
| 17:14 |
|
jvwoolf left #evergreen |
| 17:15 |
jeffdavis |
There's an additional fix in bug 1913811 that you may not have yet. |
| 17:15 |
pinesol |
Launchpad bug 1913811 in Evergreen "Too many parallel open-ils.actor.user.itemsout.notices requests" [High,Confirmed] https://launchpad.net/bugs/1913811 |
| 17:16 |
jeffdavis |
(Items Out tab in patron record) |
| 17:21 |
csharp |
jeffdavis++ # nice! thanks for the reference |
| 17:21 |
|
jihpringle joined #evergreen |
| 17:22 |
jeffdavis |
berick++ # fixin' stuff |
| 17:22 |
csharp |
berick++ |
| 17:23 |
csharp |
@praise add <cameron_voice>$who you're my HERO</cameron_voice> |
| 17:23 |
pinesol |
csharp: The operation succeeded. Praise #21 added. |
| 17:23 |
csharp |
@praise 21 berick |
| 17:23 |
* pinesol |
<cameron_voice>berick you're csharp's HERO</cameron_voice> |
| 17:23 |
mmorgan |
csharp++ |
| 17:23 |
mmorgan |
berick++ |
| 17:25 |
csharp |
"when csharp was in Egypt's land... let my csharp go..." |
| 17:25 |
csharp |
so many quotables from that movie |
| 17:32 |
|
dbwells joined #evergreen |
| 17:38 |
|
mmorgan left #evergreen |
| 17:41 |
|
jihpringle joined #evergreen |
| 18:01 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 22:59 |
|
Cocopuff2018 joined #evergreen |
