| Time |
Nick |
Message |
| 05:01 |
pinesol_green |
News from qatests: Test Success <http://testing.evergreen-ils.org/~live> |
| 05:51 |
|
jonadab joined #evergreen |
| 07:11 |
|
rjackson_isl joined #evergreen |
| 08:29 |
|
collum joined #evergreen |
| 08:55 |
|
bos20k joined #evergreen |
| 09:11 |
|
mmorgan joined #evergreen |
| 09:13 |
|
yboston joined #evergreen |
| 09:14 |
|
terran joined #evergreen |
| 09:22 |
|
Dyrcona joined #evergreen |
| 09:23 |
|
kmlussier joined #evergreen |
| 09:36 |
|
bos20k joined #evergreen |
| 09:41 |
kmlussier |
Good morning #evergreen! |
| 09:42 |
terran |
Good morning, kmlussier! |
| 09:43 |
mmorgan |
Good morning! |
| 10:12 |
phasefx |
good morning |
| 10:22 |
Bmagic |
@coffee |
| 10:23 |
* pinesol_green |
brews and pours a cup of Espresso Nuevo, and sends it sliding down the bar to Bmagic |
| 10:23 |
Bmagic |
no no, coffee please, not espresso |
| 10:23 |
Bmagic |
try again @coffee |
| 10:24 |
Bmagic |
@coffee [someone] |
| 10:24 |
* pinesol_green |
brews and pours a cup of Kenya Peaberry Kirinyaga Kii, and sends it sliding down the bar to Shae_ |
| 10:49 |
csharp |
noticed this message this morning post upgrade in the osrfwarn.log: |
| 10:49 |
csharp |
Unable to connect to ::1:7080 / gapines.org for added content lookup for 4727214: Net::HTTP::NB: Bad hostname '::1' |
| 10:49 |
csharp |
seems like Net::HTTP::NB isn't IPv6 friendly? |
| 10:49 |
berick |
csharp: there's a bug for that.. |
| 10:50 |
csharp |
it was probably happening before, but just noticing now.. oh |
| 10:50 |
berick |
csharp: https://bugs.launchpad.net/evergreen/+bug/1691473 |
| 10:50 |
pinesol_green |
Launchpad bug 1691473 in Evergreen "internal added content lookups fail if using non-standard port" [Medium,Fix released] |
| 10:51 |
csharp |
berick++ # thanks |
| 10:51 |
csharp |
my google-fu was not up to the task |
| 10:53 |
csharp |
well, hmm - I have already enabled that after seeing port 80 on the error originally |
| 10:53 |
StomproJ |
/nick Stompro |
| 10:53 |
csharp |
but it's the Bad hostname '::1' part that I'm trying to suss out now |
| 10:54 |
csharp |
we don't have any local AC so probably not a major issue for us |
| 10:54 |
berick |
csharp: it connect to localhost for all apache-driven added content lookups |
| 10:55 |
csharp |
oh |
| 10:55 |
csharp |
hmm - well why is added content working at all? |
| 10:56 |
berick |
this sounds a lot like the issue I had before where nginx would toggle between ipv4 and ipv6 connections to apache |
| 10:56 |
csharp |
that sounds right |
| 10:57 |
berick |
i fixed it locally by proxying to 127.0.0.1 instead of localhost |
| 10:57 |
csharp |
that would explain why AC is loading |
| 10:58 |
berick |
but apache is listenting on :::7080 on my server |
| 11:01 |
berick |
hm, no, it's already trying to connect to 7080, so the proxy is not the issue (i don't think) |
| 11:04 |
berick |
net::http* supports ipv6 |
| 11:13 |
berick |
csharp: OK, so net::http:nb wants IPv6 addresses wrapped in brackets. explains why it's unhappy. and it's probably using IPv6 because nginx open the connection to apache via IPv6 addr (intermittently) |
| 11:14 |
berick |
so proxying to 127.0.0.1 would probably resolve it |
| 11:15 |
berick |
but we should make EGCatLoader/Record smarter too so it doesn't croak on IPv6 addrs |
| 11:16 |
berick |
e.g. Net::HTTP->new(Host => "[::1]") works but not Net::HTTP->new(Host => "::1") |
| 11:16 |
Dyrcona |
A lot of things want IPv6 addresses wrapped in brackets. |
| 11:19 |
csharp |
still seeing it after s/localhost/127.0.0.1/g in osrf-ws-http-proxy - I'm skeered of fiddling with apache too much on upgrade morning :-) |
| 11:23 |
berick |
huh |
| 11:25 |
Dyrcona |
csharp: I assume you restarted nginx or apache or both as necessary? |
| 11:31 |
csharp |
yep - restarted both - nginx, then apache |
| 11:32 |
csharp |
not sure how my $ac_addr = $ENV{SERVER_ADDR}; works |
| 11:33 |
* csharp |
doesn't appear to have a $SERVER_ADDR environment variable defined |
| 11:34 |
berick |
apache sets that for us |
| 11:35 |
|
abowling1 joined #evergreen |
| 11:36 |
berick |
or maybe mod_perl i forget |
| 11:37 |
dbs |
server internals according to http://httpd.apache.org/docs/current/mod/mod_rewrite.html |
| 11:38 |
csharp |
dbs++ |
| 11:39 |
|
jonadab joined #evergreen |
| 11:40 |
dbs |
I think berick++ deserves a lot more karma! |
| 11:40 |
csharp |
berick++ |
| 11:46 |
* csharp |
disables IPv6 and watches the errors disapper |
| 11:46 |
csharp |
disappear, even |
| 11:59 |
berick |
csharp: disabled it in the Apache config? |
| 12:04 |
|
jihpringle joined #evergreen |
| 12:26 |
kmlussier |
@dessert [someone] |
| 12:26 |
* pinesol_green |
grabs some wild Alaskan rhubarb pie for rlefaive |
| 12:44 |
csharp |
berick: on the server via sysctl (ubuntu 14.04) |
| 12:45 |
* csharp |
has never tried rhubarb - pie or otherwise |
| 12:46 |
gmcharlt |
csharp: wow, you've missed out! |
| 12:48 |
terran |
Every time I've tried rhubarb pie I've felt like I was the victim of a culinary prank. |
| 12:49 |
gmcharlt |
collab branch for the 3.0-beta release notes is now started: collab/gmcharlt/release-notes-3-0-beta |
| 12:50 |
* mmorgan |
prefers strawberry rhubarb, preferably with vanilla ice cream :) |
| 12:51 |
jonadab |
Strawberry rhubarb is good; but best pie is pumpkin pie. But only if you make it right. (Most important rule: if you don't beat the eggwhites separately and stir them in gently _last thing_, you're doing it wrong.) |
| 13:00 |
csharp |
jonadab: so more of a pumpkin chiffon? |
| 13:00 |
* csharp |
likes super dense pumpkin pie |
| 13:01 |
csharp |
and only, like, once or twice a year |
| 13:11 |
terran |
csharp: so more like pumpkin cheesecake? |
| 13:13 |
* jonadab |
only knows the word "chiffon" in the context of sewing. |
| 13:15 |
kmlussier |
My father used to make a great lemon chiffon pie. |
| 13:16 |
* kmlussier |
also prefers strawberry rhubarb, preferably made by somebody other than me. |
| 13:16 |
kmlussier |
I tried making it once, and it was terrible. |
| 13:24 |
jonadab |
I make a decent pink grapefruit meringue pie, if I'm allowed to cheat on the crust with those Pillsbury pre-made ones, just roll it a bit thinner and go. |
| 13:25 |
jonadab |
If I make the crust from scratch, it's mediocre at best. |
| 13:27 |
csharp |
jonadab: chiffon is sort of lighter, more whipped |
| 13:52 |
csharp |
berick: does nginx proxying break OPAC IP redirection? (e.g. lib_ips.txt) |
| 13:52 |
* csharp |
just received a complaint about it |
| 13:54 |
berick |
csharp: sounds familiar... |
| 13:54 |
Dyrcona |
csharp: What was the gist of the complaint? |
| 13:56 |
csharp |
"our OPACs aren't redirecting" is the gist of it |
| 13:56 |
Dyrcona |
OK. I thought that's more or less what it would be. |
| 13:57 |
csharp |
I would expect apache would only see all requests originating from 127.0.0.1 in our case |
| 13:57 |
berick |
csharp: i can't find anything. i'm not super surprised, though. mod-rpaf may not be causing REMOTE_ADDR to get updated |
| 13:59 |
berick |
hm, github page suggests that it does |
| 13:59 |
csharp |
ah |
| 14:01 |
berick |
if apache log level is at debug, it logs a message 'Redirector sees client frim <ip>' |
| 14:01 |
berick |
[sic] |
| 14:02 |
|
collum joined #evergreen |
| 14:02 |
berick |
could be an issue with the directed-to port |
| 14:02 |
rlefaive |
Thanks kmlussier, sorry i’m late to the party but it is definitely pie season & rhubarb is my favourite. it does need strawberry though. |
| 14:02 |
Dyrcona |
Does github look messed up for anyone else? |
| 14:02 |
csharp |
on another nginx subject, I'm seeing lots of "upstream timed out (110: Connection timed out) while reading response header from upstream, client:" and I'm wondering whether that's "normal" |
| 14:02 |
berick |
Dyrcona: yeah |
| 14:03 |
berick |
csharp: do you have a proxy_read_timeout value? |
| 14:04 |
jeffdavis |
csharp: we added "proxy_read_timeout 180s;" to our nginx conf |
| 14:04 |
jeffdavis |
jinx :) |
| 14:04 |
berick |
jeffdavis: oh good, i wasn't entirely sure that was the same thing |
| 14:04 |
csharp |
proxy_read_timeout 1h; is what was in the default conf |
| 14:05 |
berick |
1 hour? |
| 14:05 |
csharp |
# Raise the default nginx proxy timeout values to an arbitrarily |
| 14:05 |
csharp |
# high value so that we can leverage osrf-websocket-translator's |
| 14:05 |
csharp |
# timeout settings. |
| 14:05 |
csharp |
yeah |
| 14:05 |
jeffdavis |
csharp: that's just for the websocket translator though |
| 14:05 |
berick |
yeah |
| 14:05 |
csharp |
oh |
| 14:05 |
csharp |
oh - then no |
| 14:05 |
berick |
it defaults to 60 seconds otherwise |
| 14:06 |
csharp |
so it would go under the "location / {" block, right?? |
| 14:06 |
rlefaive |
Dyrcona yeah, that looks like octocat had a freakout and tore the curtains down |
| 14:06 |
csharp |
(blocks) |
| 14:06 |
jeffdavis |
that's where we have it, yeah |
| 14:06 |
csharp |
jeffdavis: cool thanks |
| 14:09 |
Dyrcona |
csharp | berick: It looks like you need to set RPAF_SetHostName to On, and the proxy needs to send the X-Host or X-Forward-Host headers for the Host to get passed correctly. |
| 14:10 |
Dyrcona |
X-Forwarded-Host: rather. |
| 14:12 |
berick |
Dyrcona: that should be true w/ the default/sample configs |
| 14:12 |
* csharp |
notices commented out RPAFheader X-Real-IP in the default rpaf.conf |
| 14:12 |
Dyrcona |
berick: OK. |
| 14:13 |
* csharp |
needs to stop tempting fate by experimenting on the live server |
| 14:13 |
csharp |
we've had the smoothest go-live day in recent memory |
| 14:13 |
berick |
csharp: 2.12? |
| 14:13 |
csharp |
I would hate to spoil it with my tinkering :-) |
| 14:13 |
csharp |
berick: yeah |
| 14:13 |
berick |
awesome |
| 14:13 |
csharp |
2.12.4 |
| 14:14 |
csharp |
would've gone with .5, but we didn't have time to fully test it |
| 14:14 |
csharp |
will probably backport some of those patches |
| 14:14 |
berick |
csharp++ # expanding the nginx testing |
| 14:14 |
berick |
s/testing/using and fixing/ |
| 14:15 |
csharp |
berick++ Dyrcona++ jeffdavis++ # sharing expertise ;-) |
| 14:24 |
terran |
For everybody who got to go to Hood River for the Evergreen Conference: http://www.wweek.com/news/2017/09/05/multnomah-falls-engulfed-in-flames-as-out-of-control-wildfire-races-west-through-gorge/ |
| 14:26 |
Bmagic |
omg |
| 14:27 |
csharp |
terran: that's terrible |
| 14:33 |
|
b_bonner left #evergreen |
| 14:38 |
csharp |
uncommenting RPAFheader X-Real-IP has not fixed it |
| 14:38 |
berick |
csharp: rpaf's not updating REMOTE_ADDR |
| 14:38 |
berick |
heh, i tried that too |
| 14:38 |
csharp |
I'll have to transfer everything to a test server and enable debug logs to do much more |
| 14:38 |
berick |
my $user_ip = $ENV{'HTTP_X_FORWARDED_FOR'} || $ENV{REMOTE_ADDR}; |
| 14:39 |
berick |
that just worked on my test server |
| 14:39 |
csharp |
berick: where did you add that? |
| 14:40 |
csharp |
oh - I see |
| 14:40 |
csharp |
Redirect.pm |
| 14:40 |
pastebot |
"berick" at 64.57.241.14 pasted "for csharp" (27 lines) at http://paste.evergreen-ils.org/778 |
| 14:40 |
|
terran joined #evergreen |
| 14:41 |
csharp |
berick: any changes to rpaf.conf? or should it Just Work™ |
| 14:41 |
berick |
no rpaf changes. that's basically bypassing rpaf |
| 14:41 |
csharp |
ok good |
| 15:01 |
csharp |
still not working for me, but I'm going to have to put it aside for now |
| 15:01 |
csharp |
berick++ |
| 15:01 |
csharp |
I'll set up a test server to experiment with |
| 15:34 |
|
sandbergja joined #evergreen |
| 15:54 |
gmcharlt |
reminder - the next deveopment meeting is tomorrow at 15:00 EDT / 12:00 PDT |
| 15:54 |
gmcharlt |
agenda here: working/user/cesardv/lp1145213-bib_merge_func_merge_record_assets-signoff |
| 15:54 |
berick |
csharp: an alternative: my $user_ip = $apache->headers_in->{'X-Real-IP'}; |
| 15:54 |
gmcharlt |
er, not there |
| 15:54 |
gmcharlt |
HERE: https://wiki.evergreen-ils.org/doku.php?id=dev:meetings:2017-09-06 |
| 15:54 |
pinesol_green |
[evergreen|Jason Boyer] LP1714589: Use Explicit Definition for aacs - <http://git.evergreen-ils.org/?p=Evergreen.git;a=commit;h=fbda8c2> |
| 15:54 |
berick |
csharp: that assumes nginx is passing the X-Real-IP header (which it does in the sample osrf config) |
| 16:18 |
|
sandbergja joined #evergreen |
| 17:00 |
pinesol_green |
News from qatests: Test Success <http://testing.evergreen-ils.org/~live> |
| 17:05 |
|
mmorgan left #evergreen |
| 17:49 |
|
kmlussier joined #evergreen |
| 18:05 |
|
Jillianne joined #evergreen |
