Time |
Nick |
Message |
03:15 |
|
jamesrf joined #evergreen |
03:36 |
|
jamesrf joined #evergreen |
05:01 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
05:27 |
|
jamesrf joined #evergreen |
05:37 |
|
jamesrf joined #evergreen |
05:59 |
|
jamesrf joined #evergreen |
07:00 |
|
agoben joined #evergreen |
07:11 |
|
rjackson_isl joined #evergreen |
07:41 |
|
bdljohn joined #evergreen |
08:02 |
|
Dyrcona joined #evergreen |
09:05 |
|
terran joined #evergreen |
09:13 |
Dyrcona |
Well, that went amazingly well. I just installed O/S updates on all of our Evergreen systems, adjusted our PostgreSQL configurations, and rebooted everything. |
09:14 |
Dyrcona |
Only hitch was I had to manually bring up the ZFS pools on both db servers. |
09:14 |
Dyrcona |
Not bad for a "holiday." |
09:48 |
|
stephengwills joined #evergreen |
09:49 |
|
bdljohn joined #evergreen |
09:51 |
stephengwills |
i’m reverse eng. the reporter today to try and discover why some folders that are shared consortium wide by a user are visible to other users but other folders, similarly shared using ‘manage folders’ are not visible between the same two users. any suggestions what I should be chasing down? |
10:06 |
Dyrcona |
abneiman++ # for splitting hairs |
10:06 |
abneiman |
:) |
10:07 |
Dyrcona |
me++ # My configuration changes appear to have worked. The load averages on my db server seem to be down compared to what they have been recently. |
10:08 |
Dyrcona |
Of course, that could just be the effects of the holiday, too. I'll give it a few days before formally declaring victory. :) |
10:20 |
Bmagic |
Dyrcona: I'd be interested in seeing your Postgres configs that seem to be making a load difference |
10:21 |
Bmagic |
It's something I've been tinkering with as well |
10:21 |
Dyrcona |
Bmagic: They are practically the same on both servers, only difference is replication setup, so I'll paste just the one for the main server. We have identical hardware. |
10:22 |
Dyrcona |
Also, we're using conf.d, so I'll just paste the file that has the changes from the defaults. |
10:22 |
Bmagic |
Dyrcona++ |
10:22 |
Dyrcona |
I think finally setting up hugepages correctly also makes a big difference. |
10:23 |
Bmagic |
disabled right? |
10:26 |
Dyrcona |
Nope. Hugepages i on. |
10:26 |
Dyrcona |
https://pastebin.com/hvmz80v4 |
10:27 |
Dyrcona |
You disable transparent hugepages, but configure hugepages based on your RAM use. |
10:28 |
Dyrcona |
Spoke too soon, there's a search going for 8.[something] seconds. |
10:28 |
Dyrcona |
heh. wrong tab. :) |
10:34 |
Dyrcona |
I may have overestimated how many hugepages we need, too. |
10:36 |
Dyrcona |
Bmagic: https://www.postgresql.org/docs/9.6/kernel-resources.html#LINUX-HUGE-PAGES |
10:38 |
Dyrcona |
I didn't actually do the first steps on that page. I estimated based on the previous shared buffer size and the value of head -1 $PGDATA/postmaster.pid at that time. |
10:39 |
Dyrcona |
@monlogue |
10:39 |
pinesol |
Dyrcona: did you finish your beer? |
10:39 |
Dyrcona |
Nope. I didn't. |
10:39 |
Dyrcona |
:) |
10:41 |
bshum |
Haha |
10:41 |
Dyrcona |
I did finish my tea, though. |
10:42 |
Dyrcona |
@tea |
10:42 |
* pinesol |
brews and pours a pot of Wuyi Mountain Big Red Robe, and sends it sliding down the bar to Dyrcona (http://ratetea.com/tea/verdant/wuyi-mountain-big-red-robe/7311/) |
10:43 |
Dyrcona |
Well, that would be nice, but I'm going to have to settle for Newman's Own Organic Black. |
10:50 |
Bmagic |
sorry, someone is taking my time right now |
10:52 |
Dyrcona |
NP. I don't expect a response, you can read the things that I shared at your leisure. :) |
11:07 |
|
wcuster joined #evergreen |
12:13 |
Bmagic |
I'm getting this error quite a bit on our 3.1.9 machine:CALL: open-ils.cstore open-ils.cstore.direct.config.usr_setting_type.search.atomic {"name":[]} followed by<500> Severe query error -- see error log for more details |
12:13 |
Bmagic |
Appears in irc logs here http://irc.evergreen-ils.org/evergreen/2017-10-17#i_330455 |
12:30 |
Dyrcona |
Bmagic: Something is searching for usr_settings with an empty array, thus the sever querry error. It would be helpful if you can figure where in Evergreen it's coming from. |
12:31 |
Bmagic |
I agree, lol. I was just contemplating that |
12:31 |
Bmagic |
seems to be around login time but I'm not certain |
12:31 |
Dyrcona |
The query ends up looking something like: 'select * from actor.usr_setting where name in ();' |
12:33 |
Dyrcona |
Bmagic: If that's the case, I'd expect it to also inlcude a usr: id in the query where id is the actor.usr.id of the person just logged in. |
12:33 |
Dyrcona |
Can you tell if it is staff or OPAC/patron login? |
12:35 |
Bmagic |
Staff is what I meant but no, I haven't made the determination. I was tailing the log while logging in and saw it but that's not definitive |
12:36 |
Dyrcona |
Ah, wait....I got the wrong table. I'll bet something is trying to look up all of the setting types. |
12:37 |
Dyrcona |
Bmagic: The culprit will be in the web staff client code, somewhere. How's that for helpful? :) |
12:37 |
Bmagic |
:) |
12:37 |
Bmagic |
it's obviously not causing a crucial issue |
12:37 |
Bmagic |
which is what I was hunting for at the time of discovery |
12:38 |
Dyrcona |
But.... No. The web staff client can't use cstore directly, so it's something that gets called by the web staff client. |
12:38 |
Bmagic |
I was trying to find the root cause of the web staff client displaying the loading swirl while opening the iframed catalog |
12:38 |
Dyrcona |
Well, there are a lot of things like that in Evergreen....Errors that show up in the logs but don't seem to actually break anything. |
12:39 |
Bmagic |
yeah, one of my favorites: the enormous dumps of errors when Evergreen starts up complaining about tables missing in the database money schema |
12:39 |
Bmagic |
shows up in postgres logs |
12:45 |
Dyrcona |
Bmagic: This is a possible culprit: open-ils.actor.event_def.opt_in.settings |
12:47 |
Dyrcona |
But, there's a check in there that should prevent such a query from being run, though if the search for the opt in settings returns an error.... |
12:52 |
Dyrcona |
Bmagic: This query might help: select cust.name from action_trigger.event_definition atevdef join config.usr_setting_type cust on atevdef.opt_in_setting = cust.name where active; |
12:55 |
|
bdljohn joined #evergreen |
12:57 |
Dyrcona |
Yeah, a call to open-ils.actor.event_def.opt_in_settings looks like the most likely culprit. |
12:59 |
* gmcharlt |
clams 1148 in the name of latchkey kids everywhere! |
13:03 |
Dyrcona |
gmcharlt: You're going to use a new upgrade number and not just replace the previous script? (I'm OK with that, but just confirming.) |
13:04 |
Dyrcona |
latchkey_kids++ # I was one. |
13:04 |
gmcharlt |
Dyrcona: correct; too much time has passed since the introduction of the bug for anybody doing dev on master |
13:04 |
Dyrcona |
gmcharlt++ |
13:04 |
* Dyrcona |
pinesol identify Dyrcona Kk78jkl |
13:04 |
gmcharlt |
patching the original update IMO would have been more the thing to do if the problem had been caught within a day or so of the original bug |
13:04 |
Dyrcona |
Oops. |
13:04 |
gmcharlt |
today we are all now Dyrcona! |
13:05 |
pinesol |
[evergreen|Chris Sharp] LP#1812733 - Repair actor.usr_delete function. - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=c819ee4> |
13:05 |
pinesol |
[evergreen|Galen Charlton] LP#1812733: put in correct version of actor.usr_delete() - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=84b19c0> |
13:05 |
pinesol |
[evergreen|Galen Charlton] LP#1812733: stamp schema update - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=eb2d382> |
13:07 |
Dyrcona |
:) |
13:07 |
* Dyrcona |
changed his password. :) |
13:07 |
Dyrcona |
@band add Latchkey Kids |
13:07 |
pinesol |
Dyrcona: Band 'Latchkey Kids' added to list |
13:09 |
Dyrcona |
Also, IRC_logs++. I searched my private logs back to 2017 to find where gmcharlt told me one way to load our database into memory. That came in handy this morning. |
13:09 |
gmcharlt |
yay! |
13:11 |
Dyrcona |
I would have tried the pg_prewarm extension, but that looked like more work, creating the extension, and then prewarming every table. |
13:23 |
pinesol |
[evergreen|Jeff Davis] LP#1752665: suppress located URIs in holdings view - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=5117c42> |
13:27 |
pinesol |
[evergreen|Galen Charlton] LP#1726568: expose circ lib and owning lib fields to spine/pocket labels - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=ec4c7e6> |
13:33 |
* gmcharlt |
claims 1149 in the name of bill collectors everywhere! |
13:39 |
pinesol |
[evergreen|Jason Stephenson] Lp 1806709: Add circ.patron.billhistory_xacts persist key to database. - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=2f1db3d> |
13:39 |
pinesol |
[evergreen|Galen Charlton] LP#1806709: stamp schema update - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=8acac46> |
13:49 |
pinesol |
[evergreen|Cesar Velez] LP#1799963 - fix catalog advanced search type default - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=a96707b> |
13:49 |
pinesol |
[evergreen|Jason Boyer] LP1799963: Adjust fix to Staff Default Adv Search - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=23a2f50> |
13:58 |
|
stephengwills joined #evergreen |
14:01 |
pinesol |
[evergreen|Jane Sandberg] LP1796225: Only auto-focus search on the home page - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=fa490cf> |
14:13 |
pinesol |
[evergreen|Jason Stephenson] Lp 1813290: Fix IAMBROWSER check in ac lineitem table. - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=d748fe9> |
14:19 |
|
stephengwills joined #evergreen |
14:21 |
pinesol |
[evergreen|Sam Link] LP1802594 - Splash Page Image Alt Text - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=51a043e> |
14:21 |
pinesol |
[evergreen|Galen Charlton] LP#1802594: (follow-up) avoid a regression on "Copy" => "Item" - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=28e35b4> |
14:45 |
Bmagic |
Dyrcona: lol |
14:45 |
Dyrcona |
@blame Latchkey Kids |
14:46 |
pinesol |
Dyrcona: I know it was you, Latchkey Kids. You broke Dyrcona's heart. You broke Dyrcona's heart. |
15:32 |
|
sandbergja joined #evergreen |
15:50 |
|
seymour joined #evergreen |
16:30 |
|
stephengwills joined #evergreen |
16:52 |
|
sandbergja joined #evergreen |
16:53 |
|
sandbergja joined #evergreen |
16:54 |
|
sandbergja_ joined #evergreen |
17:01 |
pinesol |
News from qatests: Failed Running pgTAP tests <http://testing.evergreen-ils.org/~live/test.42.html#2019-02-18T16:58:38,346652136-0500 -0> |
21:34 |
|
jeff joined #evergreen |
21:58 |
|
Christineb joined #evergreen |
22:20 |
|
sandbergja joined #evergreen |