| Time |
Nick |
Message |
| 09:01 |
|
dguarrac joined #evergreen |
| 09:16 |
|
Dyrcona joined #evergreen |
| 10:29 |
|
sandbergja joined #evergreen |
| 11:07 |
|
Christineb joined #evergreen |
| 11:45 |
csharp_ |
aaaaaaahhhhhh - the nb_bouncer fix is working great in PINES production |
| 11:49 |
berick |
csharp_: ? |
| 11:49 |
csharp_ |
bug 2113979 - in case it wasn't clear |
| 11:49 |
pinesol |
Launchpad bug 2113979 in Evergreen 3.15 "Put a hurdle in front of basic scraping" [Medium,New] https://launchpad.net/bugs/2113979 |
| 11:49 |
csharp_ |
berick: ha - I wondered if I would need to clarify |
| 11:51 |
csharp_ |
we hates the botses |
| 11:53 |
berick |
ah, gotcha |
| 11:55 |
|
abowling joined #evergreen |
| 13:35 |
eeevil |
csharp_: once the bots start running JS we'll have to add some more bumps, but(!) that should just be changes to the TT2. fingers crossed... |
| 13:37 |
jeffdavis |
thinking of adding a line to the TT2: "If you are scraping the OPAC, maybe try emailing us to ask for our data, which we're happy to share, instead of being a dick about it." |
| 13:55 |
csharp_ |
eeevil++ |
| 13:55 |
csharp_ |
jeffdavis++ |
| 13:56 |
csharp_ |
I put the scraping patch in place after angrily reading a bot owner's website saying how "polite" their bot is |
| 14:00 |
Dyrcona |
Polite bots obey robots.txt. |
| 14:11 |
Bmagic |
csharp_++ # great success |
| 14:12 |
csharp_ |
maaayyybe |
| 14:13 |
csharp_ |
having trouble with quipu ecard registration and I suspect it's because of the patch |
| 14:18 |
csharp_ |
confirmed - disabling the bot cookie require directive in eg_vhost allows ecard registration to to work |
| 14:18 |
csharp_ |
so... we need to figure out a way to generate a cookie for quipu |
| 14:18 |
csharp_ |
(or maybe other legitimate non-human users?) |
| 14:41 |
eeevil |
csharp_: it can be added to the front-door list in the perl module |
| 14:41 |
eeevil |
then it'll hand out a cookie (if we serve something at the url, that is) |
| 14:41 |
csharp_ |
ohhh |
| 14:41 |
csharp_ |
I was looking at that |
| 14:42 |
csharp_ |
eeevil: so /eg/opac/register is the path - so just add "register" to the list? |
| 14:42 |
eeevil |
you might be able to disable the cookie requirement by adding a path match on JUST the quipu url to turn off the perl variable |
| 14:43 |
eeevil |
yes, that's right. just add register as a peer to home etc |
| 14:43 |
csharp_ |
eeevil++ |
| 14:54 |
jeffdavis |
I'm finding that if I go directly to a results page with no cookie, I get redirected to nb_bounce 6-12 times before a cookie is set and my page loads - is that amount of redirecting expected? |
| 14:54 |
jeffdavis |
(testing the patch for 2113979) |
| 14:57 |
eeevil |
it's not, and we haven't seen that here ... and that doesn't actually make sense... maybe the JS is redirecting before the browser stores the cookie that came in the header? you should def be getting a set-cookie header /every/ time you hit the nb_bounce url. def interested (on the bug) in any details that might point to a cause |
| 15:00 |
JBoyer |
The Network tab of the browser console should be a big help there. Oh, jeffdavis, is that system behind multiple apache servers or just one (like a test system?) |
| 15:05 |
jeffdavis |
test system with HAProxy in front of nginx in front of Apache |
| 15:06 |
JBoyer |
I think my one / many apache server concern is no problem, I was thinking about the localhost memcached that the translator defaults to using, but that's not the cache used to store the cookie. |
| 15:07 |
JBoyer |
So I'm not sure what would cause that. Just using Chrome? |
| 15:08 |
jeffdavis |
This is in Chromium. |
| 15:08 |
JBoyer |
Is the hostname publicly reachable? |
| 15:12 |
jeffdavis |
yeah, bw.upgrade.catalogue.libraries.coop |
| 15:21 |
JBoyer |
Well, today I learned that `apt-get install chromium` does not recommend installing transitional package 'chromium-browser' (snaps, ugh.) but instead suggests installing a game named chromium-bsu. oops. |
| 15:22 |
csharp_ |
yeah, it's always been a naming conflict with whatever chromium came first :-/ |
| 15:23 |
csharp_ |
also snaps-- |
| 15:24 |
jeffdavis |
looks like I'm seeing the same behavior in Firefox fwiw, if you don't want to install games |
| 15:24 |
JBoyer |
Now to find where Google hid the "free the console from these confines" option, because how do even with this embeded thing. |
| 15:27 |
JBoyer |
So good news / bad news. Good: Chromium did do the same thing for me, though it looked like quite a few as fast as possible; Bad: I forgot to click Preserve Log :/ |
| 15:30 |
jeffdavis |
I miss that step every time |
| 15:32 |
JBoyer |
So, for all of them I only get provisoinal response headers (lots of local X-Eg-Env stuff) but not response proper (there's a .tt2 page with a link to click, etc.) |
| 15:33 |
JBoyer |
The cookie did come through at least the first time, but after that it looks like most of the 302's are a result of caching for the initial results page. |
| 15:33 |
JBoyer |
And after enough repeated loads of the same ping pong match it requests .../results from the server again and it already has the cookie its looking for. |
| 15:36 |
JBoyer |
Looks like it bounced around about 140 times for me. I'd check your cache config to make sure it matches what you're sending in your headers. |
| 15:36 |
JBoyer |
I don't know what the default is for the results page, but I'd think it would be to not cache anything. Not sure why Chromium wants to do that. |
| 15:40 |
JBoyer |
^To explain a little better, the ~140 requests to /eg/opac/results after the initial redirect had a status of "302 redirected (from disk cache)" |
| 15:49 |
csharp_ |
@decide passwd or asswd |
| 15:49 |
pinesol |
csharp_: go with asswd |
| 15:50 |
pinesol |
News from commits: LP#2113754 Precat checkouts are no longer possible for blocked patrons <https://git.evergreen-ils.org/?p=Evergreen.git;a=commitdiff;h=d438e83692c58e04b50010451b88b121547f6c2c> |
| 15:52 |
jeffdavis |
I don't think we're doing anything special with caching, AFAICT we're just going with the defaults |
| 15:53 |
JBoyer |
Asswd(TM): Schoolyard bullying for the modern branding-is-hard-lets-drop-vowels era. |
| 15:56 |
JBoyer |
Oh, that's intersting. Looks like the Expires: header is 4 seconds after Date:. I'm guessing Chromium that explains why chromium was freaking out for ~4-5 seconds. |
| 16:19 |
jeffdavis |
Location /eg/opac has ExpiresByType text/html "access plus 5 seconds" |
| 16:19 |
jeffdavis |
does something need to be expired immediately? |
| 16:25 |
jeffdavis |
or should nb_bounce add a cache-busting key to the redirect url? |
| 16:35 |
Bmagic |
csharp_: mind posting your changes for apache eg_vhost.conf? to allow quipu? |
| 16:36 |
Bmagic |
(on the bug is probably the best place for it) |
| 16:44 |
|
Dyrcona joined #evergreen |
| 18:57 |
csharp_ |
Bmagic: my first attempt didn't work |
| 18:57 |
csharp_ |
I'll have to look again tomorrow |
| 19:58 |
jeffdavis |
noting that I get an infinite redirect loop if I disable cookies |
