Evergreen ILS Website

IRC log for #evergreen, 2021-02-03

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
03:19 dbwells joined #evergreen
07:10 rjackson_isl_hom joined #evergreen
07:18 csharp @quote add <Bmagic> It's as if millions of bugs all cried out at once, and were silenced.
07:18 pinesol csharp: The operation succeeded.  Quote #213 added.
08:16 mantis1 joined #evergreen
08:21 stephengwills joined #evergreen
08:35 rfrasur joined #evergreen
08:44 mmorgan joined #evergreen
09:03 Dyrcona joined #evergreen
09:04 dbwells joined #evergreen
09:07 Bmagic csharp++ # Haha... let me see if I can use the commands to search the quotes for the other ones I authored
09:16 Bmagic @quotes Bmagic
09:16 pinesol Bmagic: Have you confirmed your ISBN SPIDs with your service provider?
09:17 Bmagic @quote Bmagic
09:17 pinesol Bmagic: Down time is a fact of business when you're a poor 501c3 corporation.
09:17 Bmagic struck out
09:18 Bmagic Pretty sure this is a feature? Though it's not documented here https://wiki.evergreen-ils.org/doku.​php?id=community:using-pinesol_green
09:18 Bmagic where's the source code for pinesol?
09:19 Bmagic @quote search Bmagic
09:19 pinesol Bmagic: 4 found: #149: "< Bmagic> my favorite viruses are the ones...", #172: "< Bmagic> I like to keep my systems at a power...", #179: "<Bmagic> It's amazing how often 30 days comes...", and #213: "<Bmagic> It's as if millions of bugs all cried..."
09:19 Bmagic bingo
09:19 Bmagic @quote #149
09:19 pinesol Bmagic: http://www.firstpersontetris.com/
09:20 Bmagic @quote 149
09:20 pinesol Bmagic: Your computer account is overdrawn. Please reauthorize.
09:20 Bmagic @quote show #149
09:20 pinesol Bmagic: NOBODY EXPECTS THE SPANISH INQUISITION!
09:20 Bmagic man I suck at this
09:20 Bmagic @quote help
09:20 pinesol Bmagic: https://i.imgur.com/m8EySrW.gifv
09:21 Bmagic I clicked on that thinking it would help me. I was wrong
09:22 Bmagic @help quote
09:22 pinesol Bmagic: Error: There is no command "quote". However, "Quote" is the name of a loaded plugin, and you may be able to find its provided commands using 'list Quote'.
09:22 Bmagic @help Quote
09:22 pinesol Bmagic: Error: There is no command "quote". However, "Quote" is the name of a loaded plugin, and you may be able to find its provided commands using 'list Quote'.
09:22 Bmagic @list Quote
09:22 pinesol Bmagic: add, change, get, random, remove, replace, search, and stats
09:23 Bmagic @quote get #149
09:23 pinesol Bmagic: (quote get [<channel>] <id>) -- Gets the quote with id <id> from the quote database for <channel>. <channel> is only necessary if the message isn't sent in the channel itself.
09:23 Bmagic @quote get 149
09:23 pinesol Bmagic: Quote #149: "< Bmagic> my favorite viruses are the ones claiming to be the cure" (added by csharp at 10:32 AM, March 22, 2016)
09:24 Bmagic let this be a lesson to everyone! - and now, I'm editing the wiki
09:25 Dyrcona Bmagic: You can do all of that in a private conversation with pinesol so as not to spam the channel.
09:25 Bmagic dang it
09:25 Dyrcona @quote get 149
09:25 pinesol Dyrcona: Quote #149: "< Bmagic> my favorite viruses are the ones claiming to be the cure" (added by csharp at 10:32 AM, March 22, 2016)
09:25 Bmagic sorry. I'll be sure and mention that in my wiki edit
09:27 Dyrcona You need to add the channel to the command when doing a private chat because the bot keeps track of quotes, etc. by channel.
09:27 Dyrcona @dunno
09:27 pinesol Dyrcona: Have you tried throwing it across the room?
09:27 Bmagic yep - the help message above mentioned that
09:47 csharp I'm mostly interested to see all the @dunno entries I'd forgotten about :-)
09:51 Dyrcona So, I have to figure out why action.emergency_closing_stage_2_hold is so slow. I cancelled two yesterday that ran for a day, and 1 this morning.
09:51 Dyrcona I'm gonna open a local ticket so I don't forget about it.
10:09 dbwells joined #evergreen
10:14 Dyrcona Does anyone else think that the emergency closing code is too slow? I'm asking because maybe it's just us.
10:19 pinesol [evergreen|Garry Collum] LP1868147-Titles in carousel run together in mobile display. - <http://git.evergreen-ils.org/?p=​Evergreen.git;a=commit;h=ef55734>
10:20 dbwells joined #evergreen
10:23 alynn26 joined #evergreen
10:32 dbwells joined #evergreen
10:43 csharp Dyrcona: that sounds familiar, but I never dug in
11:06 dbwells joined #evergreen
11:06 dbwells_ joined #evergreen
11:11 Dyrcona So, search is busted on my training server after an upgrade from 3.2.10 to 3.5.2. I've tried a reingest (with pingest.pl), checked that the opac templates (they look OK), checked the logs for errors (nothing looks related to search), and I've verified that we have all of the upgrades from upgrade directory installed.
11:11 Dyrcona Anyone got any ideas of what to try next?
11:15 csharp check activity logs for the searches and trace them through the osrfsys log?
11:15 csharp (assuming those are broken out)
11:21 Dyrcona Well, I have more information since I typed that. Others have updated our internal ticket.
11:22 Dyrcona Looks like search is working in the "experimental catalog." If you do a search that returns a single hit, it works.
11:22 Dyrcona What we get in the results is a list of "blank" results, so it is looking like a possible template issue.
11:24 Dyrcona I have removed our local templates and checked out those from origin/tags/rel_3_5_2 and those needed for curbside and stripe backports. I'll restore our templates one at a time until search breaks, assuming it's working, now.
11:26 Dyrcona Well, nope. That didn't fix it.
11:28 Dyrcona Is anyone running 3.5.2 in production?
11:35 Dyrcona Here's a screen shot with our customizations removed: https://drive.google.com/file/d/1MINHLMKn​aPKx10KZ98RgyA6v0ay8OW7W/view?usp=sharing
11:39 Dyrcona And, here's one with our customizations: https://drive.google.com/file/d/1_2o2k7wy​-t5WJnxrGg93OEmY_VvR_WLW/view?usp=sharing
11:51 Dyrcona So, I'm going to do a git clean -x -f -d and rm -rf /openils/var/templates, and then do a clean install. IIRC, I did rm -rf /openils before installing everything fresh. I suppose I could also try deleting /usr/local/share/perl/5.26.1/OpenILS.
12:00 Dyrcona At 11:22:10 when I said search was working in the experimental catalog, I meant that it was getting multiple results. The single hit statement was about the TTOPAC. Also, the experimental catalog information is anecdotal. I haven't checked it, myself.
12:00 Dyrcona And, it's lunch time.
12:09 jihpringle joined #evergreen
12:13 Bmagic Dyrcona: making the templates Evergreen stock is a good first step I would say. Next, I would reinstall OpenSRF/Evergreen. Nothing in the logs to go on?
12:13 Bmagic Dyrcona: yes, the emergency closing code is slow. I've seen it run for days sometimes
12:18 Cocopuff2018 joined #evergreen
12:27 sandbergja_ joined #evergreen
12:35 jeffdavis We're somewhere between 3.5.1 and 3.5.2 in production if it helps.
12:53 jihpringle joined #evergreen
12:59 Dyrcona Bmagic: I already reverted to stock templates and it was the same I shared two screen shots, in case anyone missed them.
13:00 Bmagic I saw them, I was agreeing with your path
13:00 Bmagic check postgres logs - might be a missing function
13:09 alynn26_away joined #evergreen
13:11 JBoyer Dyrcona, this may or may not help, but maybe make sure fm_IDL.xml was actually updated? I remember seeing an issue like yours a couple years ago but don't recall the cause.
13:11 JBoyer I don't know how that file could be out of date but I've seen it happen in the past.
13:21 Dyrcona Despite my best efforts at comparing the upgrade log with the upgrade scripts in the Open-ILS/src/sql/Pg/upgrade directory, it looks like some upgrades are missing: https://drive.google.com/file/d/1QMg6vIK7​dAC2E-_epc4jVoN4zzPX9P_G/view?usp=sharing
13:22 Dyrcona I wonder if some of the 3.3. or 3.4 upgrades failed to make it into 3.5?
13:25 Bmagic Dyrcona: I think you're getting close!
13:31 Dyrcona I'm not so sure because the missing numbers are all over 1200, but I'll check them against 3.3 and 3.4 anyway.
13:33 Dyrcona None of the missing upgrades appear in rel_3_3 nor in rel_3_4.
13:36 Dyrcona So maybe code made it into 3.5 but the db upgrades were missed? However, jeffdavis says they're on 3.5, so that's not likely to be the problem.
13:38 collum joined #evergreen
13:41 Dyrcona All right, so it looks like the missing relations are local additions, and none of them have anything to do with search.
13:42 Dyrcona The two errors about composite types are the only ones that look relevant.
13:43 Dyrcona WTF? SELECT * FROM search.search_result AS x WHERE 1=0;
13:48 Dyrcona Wait a minute. The queries for the missing tables are the same. Is this related to fieldmapper? Guess, I'll listen to JBoyer and see if the IDL isn't hosed.
13:52 Dyrcona xmllint reports no errors.
13:54 sandbergja_ joined #evergreen
14:03 Dyrcona Oddly enough, I can't find the missing relations in our production databases, either, but they don't spit out those errors.
14:07 Dyrcona The code producing SELECT * FROM %s AS x WHERE 1=0; has been there since 2010.
14:10 Dyrcona "Curiouser and curiouser," said Alice.
14:12 Dyrcona Now, I can't find those tables in our IDL, but I swear they were there before...
14:12 csharp @who moved Dyrcona's tables while listening to the new album from [band]?
14:12 pinesol awitter moved Dyrcona's tables while listening to the new album from Sentient Peach.
14:13 Dyrcona Heh!
14:14 csharp @praise [band]
14:14 * pinesol Huge Bean Weevil is the very model of a modern major hacker
14:14 Dyrcona Turns out I accidentally included an extra character when searching via the editor.
14:18 Dyrcona diffing our custom IDL with the one from rel_3_5 doesn't turn up anything striking. We added the acsp object for curbside, we have an extra link in 1 object (that should be added to Lp, btw), an extra blank and our custom sources at the "end."
14:18 Dyrcona Both IDLs come out as valid XML when I check them.
14:25 Dyrcona The composite types mentioned in the pgerrors.txt have been in the IDL since 2009.
14:25 Dyrcona I'm stumped.
14:30 jeffdavis Dyrcona: there are a couple of test scripts for search in Open-ILS/src/support-scripts/test-scripts/ - do they show anything interesting?
14:30 jeffdavis my next guess would be an issue with visibility attrs
14:37 Dyrcona jeffdavis: I haven't run them.
14:39 Dyrcona I don't think the problem is really search or the database, because it works with the experimental Angular catalog, but I'll give those scripts a shot. I still think it has to be the templates or something in the Perl.
14:57 Dyrcona So, query_tests.pl returns results on some of the searches. Naturally, many of them return nothing because it looks like it is written for the concerto dataset.
15:02 Dyrcona The perl modules all check out. Perl -c reports "syntax OK" on all of them.
15:02 Dyrcona Doesn't mean that there aren't bugs.
15:07 csharp "your horrible code has the proper syntax"
15:09 Dyrcona If Perl can be said to have syntax. :P
15:10 csharp TMTOWTDI can definitely be a weakness
15:29 Bmagic lol
15:30 Dyrcona TimToady is a registered nick. :)
15:31 Dyrcona So, how do I use the alternate catalog/eg2 on 3.5?
15:31 khuckins_ joined #evergreen
15:31 Dyrcona I logged in at eg2/en-US/staff but got redirected to the AngularJS staff client.
15:33 malexander joined #evergreen
15:33 JBoyer There may be an OUS gating it from appearing in the Cataloging menu, not sure.
15:34 JBoyer Unless that was 3.4...
15:34 Dyrcona Hey! Waitaminute! After doing that, I'm getting search results in the staff client, but still none in the regular OPAC.
15:34 Dyrcona Maybe that's what John meant in the ticket and I misunderstood?
15:34 Dyrcona Still doesn't tell me much.
15:36 Dyrcona Ah, wait. Now, when I search for the whole consoritum, with 10 results, and not just my branch, with 2, I get blank results....
15:40 Dyrcona It's working that way in the training OPAC, after logging out, closing the window, etc.
15:41 Dyrcona So, it could be related to db performance or something. Think I'll shut it all down, install updates, reboot and vaccum the db.
15:41 mantis1 left #evergreen
16:00 jvwoolf joined #evergreen
16:02 Dyrcona Well, isn't that nice... => /var/lib/postgresql is using 86.5% of 511.73GB
16:04 Dyrcona Turns out some of those missing relations are not custom tables/views. I'll have to look into that tomorrow.
16:05 csharp could be holding onto xlog for some reason? unless it's legit that big
16:25 Dyrcona Our production database is 390GB after a full vacuum.
16:26 Dyrcona This one is a copy with a couple of tables truncated.
16:27 Dyrcona I don't think autovacuum helps us all that much, tbh. The production database is 413GB at the moment. I'm pretty sure that most of the extra 20GB is dead space.
16:38 Dyrcona Last time that I full vacuumed production, df indicate it was using something like 796GB and afterward is was 390GB.
16:46 csharp Dyrcona: yeah - I've not fully figured out the art of autovacuum - it's a trade off between performance and... well, performance, I guess
16:47 Dyrcona We do a vaccum analyze in production every Saturday morning.
16:52 csharp Sunday night for us
16:54 csharp okay - I'm back to having to restart opensrf every hour or so to prevent actor drone exhaustion - trying to remember where we left things... plan was to push your diagnostic fix so we can see in the browser which UIs are behaving badly, right
16:55 csharp but the only thing we saw flat-out broken was dojo stuff, which were considering irrelevant to the fix, but Acq users are impacted
16:55 csharp so the choice is acq PO loading versus drone saturation right now
16:56 * csharp meant to tag berick somewhere in there :-)
16:57 sandbergja_ joined #evergreen
16:58 Dyrcona :)
17:02 csharp gmcharlt: (or someone EOLI-ish), just FYI, looks like certbot needs a kick on yeti.esilibrary.com - cert expired on 9/29/2020
17:03 csharp I was curious about the Angular acq stuff (which I should already know more about, but don't ;-))
17:14 jeffdavis csharp: can you tell where the drone saturation is being triggered?
17:14 jvwoolf left #evergreen
17:15 jeffdavis There's an additional fix in bug 1913811 that you may not have yet.
17:15 pinesol Launchpad bug 1913811 in Evergreen "Too many parallel open-ils.actor.user.itemsout.notices requests" [High,Confirmed] https://launchpad.net/bugs/1913811
17:16 jeffdavis (Items Out tab in patron record)
17:21 csharp jeffdavis++ # nice!  thanks for the reference
17:21 jihpringle joined #evergreen
17:22 jeffdavis berick++ # fixin' stuff
17:22 csharp berick++
17:23 csharp @praise add <cameron_voice>$who you're my HERO</cameron_voice>
17:23 pinesol csharp: The operation succeeded.  Praise #21 added.
17:23 csharp @praise 21 berick
17:23 * pinesol <cameron_voice>berick you're csharp's HERO</cameron_voice>
17:23 mmorgan csharp++
17:23 mmorgan berick++
17:25 csharp "when csharp was in Egypt's land... let my csharp go..."
17:25 csharp so many quotables from that movie
17:32 dbwells joined #evergreen
17:38 mmorgan left #evergreen
17:41 jihpringle joined #evergreen
18:01 pinesol News from qatests: Testing Success <http://testing.evergreen-ils.org/~live>
22:59 Cocopuff2018 joined #evergreen

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat