| Time |
Nick |
Message |
| 05:00 |
pinesol |
News from qatests: Failed Running pgTAP tests <http://testing.evergreen-ils.org/~live/test.42.html#2019-01-31T04:57:55,332522187-0500 -0> |
| 07:01 |
|
agoben joined #evergreen |
| 07:04 |
|
rjackson_isl joined #evergreen |
| 07:20 |
JBoyer |
gsams, were you following the build steps in Hatch/installer/windows/README.adoc ? Hatch for Windows has a 2-step build process that involves building the /lib directory and filling it with .jar's, then building the installer with makensis. |
| 07:22 |
JBoyer |
So if hte lib directory is missing that just means you need to build the Java components. And once it's installed you'll almost certainly need to completely close both browsers. (Chrome seems very reluctant to load a NativeMessaging plugin except at startup. |
| 07:47 |
|
bos20k joined #evergreen |
| 07:56 |
|
jamesrf joined #evergreen |
| 08:18 |
|
Dyrcona joined #evergreen |
| 08:41 |
|
mmorgan joined #evergreen |
| 08:52 |
|
Stompro joined #evergreen |
| 09:02 |
|
terran joined #evergreen |
| 09:38 |
|
yboston joined #evergreen |
| 09:51 |
|
jvwoolf joined #evergreen |
| 09:55 |
JBoyer |
berick++ # metabib.browse_entry cleanup |
| 09:56 |
JBoyer |
I looked half an hour after cleaning it up and there are already another 20 or so. Looks like this would make a good candidate for keeping an eye on. |
| 09:59 |
berick |
i usually run about once a year |
| 10:02 |
Dyrcona |
metabib.browse_entry cleanup? |
| 10:03 |
mmorgan |
Speaking of cleaning up, does anyone regularly clean up asset.uri? |
| 10:04 |
berick |
Dyrcona: yes |
| 10:05 |
Dyrcona |
I must have missed a memo. |
| 10:05 |
rjackson_isl |
http://irc.evergreen-ils.org/evergreen/2019-01-30#i_392676 |
| 10:06 |
berick |
rjackson_isl++ |
| 10:08 |
Dyrcona |
And, that is supposed to speed things up, or what? I see that it clears dead entries. |
| 10:09 |
JBoyer |
Should make browse searches slightly faster since there aren't 2+ million unused headings in our db anymore. I was looking at pgbadger and most of the 3m+ queries were browse. |
| 10:10 |
JBoyer |
Still not "fast" but it's not miserable. (I don't know how popular it is, but is that because people don't browse or because it's too slow to use. *shrug*) |
| 10:11 |
Dyrcona |
Related: I have noticed that most of our longer running search queries are browse searches. |
| 10:18 |
Dyrcona |
The select part of that returns 5,343,301 dead entries on our production db. |
| 10:19 |
Dyrcona |
We have 14.032,444 rows in the table. That's a lot of "useless" rows. |
| 10:20 |
Dyrcona |
So, no special precautions required, you just run this whenever? |
| 10:22 |
JBoyer |
It's worth batching it with those kind of numbers since adding and editing bibs could potentially block. |
| 10:23 |
JBoyer |
otherwise there shouldn't be a problem. |
| 10:24 |
Dyrcona |
Well, I may wait and run it as part of our custom db upgrade in April. |
| 10:25 |
Dyrcona |
Is there some reason that triggers don't take care of this? |
| 10:26 |
JBoyer |
I suspect the fact that nothing actually points to bre makes it a possibility that was forgotten / ignored or fairly difficult to do. |
| 10:26 |
JBoyer |
(haven't researched it that much) |
| 10:27 |
|
Christineb joined #evergreen |
| 10:44 |
gsams |
JBoyer: Okay, assuming that step is compiling hatch, that step was the one producing JSON related errors for me yesterday. |
| 10:45 |
JBoyer |
running hatch.sh compile? |
| 10:46 |
gsams |
Yeah, but I just did that and it requires the same thing. There isn't a step that I'm seeing that speaks of the lib directory. |
| 10:47 |
JBoyer |
That's annoying. I can't really follow along at work because enterprise blah blah blah, but I'll see if I can trigger anything like that at home. |
| 10:47 |
gsams |
I must be missing something. |
| 10:47 |
JBoyer |
What Java JDK is installed and where are you building it? You don't have to build it on Windows, Linux can take care of building the Windows installer if that's easiest. |
| 10:49 |
gsams |
I've got JDK 1.8.0_201, I'm trying to build it on Windows as that is what I have to work with here, though I do have a laptop at home I could probably get it done faster on. |
| 10:49 |
gsams |
I'm also kind of new to this, so I may very well be doing something wrong. |
| 10:53 |
berick |
gsams: any reason you're not using the windows installer? |
| 10:53 |
gsams |
berick: Trying to test the firefox branch |
| 10:53 |
berick |
ah |
| 10:54 |
JBoyer |
if you just open cmd and type java what happens? There have been irritations with the java path. |
| 10:54 |
berick |
gsams: the installer (and linux compile step) fetch a json library and put it into the lib dir |
| 10:55 |
gsams |
JBoyer: I get a java usage statement |
| 10:55 |
berick |
see the top of the hatch.sh file for the path to the JSON jar file |
| 10:55 |
berick |
you'll have to manually fetch that |
| 10:55 |
berick |
the docs assume windows builds use the installer, so it's not mentioned |
| 10:55 |
berick |
and the linux script does it for you |
| 10:56 |
JBoyer |
Hey, that would be the part I was not paying attention to. (I built it in Ubuntu on Windows 10 and cp'd it to my desktop to test) |
| 10:56 |
gsams |
ha, that makes a ton of sense. Now I know why I had it the first run now. |
| 10:56 |
berick |
gsams: you may want to just use the windows installer |
| 10:57 |
berick |
and then rebuild the java bits using the updated code |
| 10:57 |
berick |
the insatller does more than just build the java |
| 10:58 |
berick |
the windows registry changes being the most irksome |
| 10:59 |
JBoyer |
berick, gsams, the java bits aren't what needs testing, FF support is specifically broken by hatch.bat. (so the easy way to test this gsams, would be install the hatch you can download now and cp hatch.bat from my branch over the one in C:\ProgramFiles (8x6)\Hatch\) |
| 10:59 |
gsams |
JBoyer: I can do that |
| 10:59 |
berick |
ah, even easier |
| 10:59 |
JBoyer |
I haven't touched Java since college and haven't gotten so brave as to try messing with anything in there. |
| 11:24 |
gsams |
Hatch is enabled in Firefox, no printers found however. Chrome is working as intended. |
| 11:26 |
berick |
JBoyer: how do the Firefox registry keys get setup? |
| 11:27 |
|
khuckins joined #evergreen |
| 11:27 |
gsams |
by building the installer with the .nsi |
| 11:27 |
gsams |
At least, that's what it looks like to me. |
| 11:28 |
berick |
ok, the installer adds the Chrome keys, wasn't sure if it added the FF keys too |
| 11:30 |
berick |
took a peak, i'm only seeing Chrome keys getting added in the current hatch.nsi |
| 11:31 |
csharp |
JBoyer's branch adds them for FF, I think? |
| 11:31 |
gsams |
csharp: Yeah, that's where i was seeing it |
| 11:31 |
berick |
in which case, the current installer won't have them |
| 11:31 |
gsams |
indeed, I'm taking a different route. I think I can still get this. |
| 11:32 |
csharp |
oh wait - iirc, FF needs you to download the addon from the addons sit |
| 11:32 |
csharp |
e |
| 11:32 |
gsams |
csharp: I did get that part |
| 11:32 |
csharp |
https://addons.mozilla.org/en-US/firefox/addon/hatch-native-messenger/ |
| 11:32 |
csharp |
cool |
| 11:33 |
berick |
so unless I'm missing something, a windows installer will need to be generated from JBoyer's branch |
| 11:34 |
berick |
the one at https://evergreen-ils.org/egdownloads/ won't work |
| 11:34 |
gsams |
That is definitely correct, and manually getting the jar file probably will fix my part of the problem. |
| 11:36 |
gsams |
okay, so I was able to compile and run the new installer. It fixed firefox, but broke chrome. |
| 11:36 |
berick |
you need the firefox registry key additions |
| 11:37 |
JBoyer |
Sorry I was away. berick is right, you do need a real installer built from my branch. I forgot about the reg keys. |
| 11:38 |
gsams |
I've built the installer from JBoyer's branch |
| 11:38 |
gsams |
I manually grabbed the jar file referenced in hatch.sh and built it. |
| 11:38 |
gsams |
It did fix firefox, but broke chrome for me. |
| 11:38 |
JBoyer |
closed and restarted both browsers? |
| 11:39 |
gsams |
Yeah |
| 11:39 |
JBoyer |
:/ |
| 11:39 |
gsams |
No printers found, hatch is available and enabled in chrome. |
| 11:39 |
berick |
gsams: oh, you built your own Windows installer exe ? |
| 11:40 |
gsams |
berick: Yeah, after you pointed out the difference in the bat and sh files I put it together and muddled through. |
| 11:43 |
gsams |
It doesn't seem right for chrome to have broken though. I may have done something else wrong. I'm going to back track a bit now that I know what was going wrong. |
| 11:44 |
JBoyer |
It's probably worth seeing what values are in these reg keys: SOFTWARE\Google\Chrome\NativeMessagingHosts\org.evergreen_ils.hatch and SOFTWARE\Mozilla\NativeMessagingHosts\org.evergreen_ils.hatch |
| 11:45 |
JBoyer |
Oh, and SOFTWARE isn't the top of those trees, it's HKEY_LOCAL_MACHINE\SOFTWARE... |
| 11:47 |
gsams |
Huh. Chrome isn't there. |
| 11:49 |
JBoyer |
Ugh, there's a WoW32 or similar key that you have to follow to get to Chrome. It's automagical for the API, but if you are using regedit you have to just know how to follow it. (FF has different rules for its key) |
| 11:49 |
gsams |
Okay, that one is there. |
| 11:50 |
JBoyer |
For Chrome, you'll have to follow WoW6432Node after SOFTWARE and before Google |
| 11:50 |
gsams |
Yeah, oddly enough when I opened regedit it was already down that rabbit hole, then I backtracked to make sure I was in the right place. |
| 11:51 |
gsams |
so chrome has the correct thing in place, but still doesnt work. |
| 11:52 |
JBoyer |
Hmm. |
| 11:53 |
JBoyer |
the path to hatch.bat in @Defaultlooks good? |
| 11:54 |
gsams |
Yeah, it's correct. |
| 11:54 |
JBoyer |
I don't suppose there's the slimmest chance that Chrome's console can help? If you open the Console tab of the dev tools the dropdown that has "top" in it may allow you to look at the extension page's logs |
| 11:55 |
berick |
chrome://extensions |
| 11:55 |
berick |
open the debug console for the extension |
| 11:56 |
JBoyer |
Oh, much easier. |
| 11:57 |
berick |
enable developer mode on the extensions page |
| 11:57 |
berick |
click extension Show Details |
| 11:57 |
berick |
should be a "background page" option |
| 11:58 |
gsams |
A lot of disconnected: Specified native messaging host not found. |
| 11:58 |
* JBoyer |
almost brought his laptop in today... |
| 11:59 |
berick |
gsams: that means the browser could not find org.evergreen_ils.hatch.json or the path inside does not point to the correct hatch.bat file |
| 12:00 |
berick |
it finds the .json file by following the registrey key values |
| 12:00 |
gsams |
I've got the json file open, and the path looks correct to me. |
| 12:00 |
gsams |
"path": "C:\\Program Files (x86)\\Hatch\\hatch.bat", |
| 12:01 |
gsams |
And the registry key value is pointing to that file |
| 12:03 |
gsams |
I'm going to back track a little, and see if I've gotten something else out of place. |
| 12:04 |
JBoyer |
May be worth uninstalling it and trying to start from a clean slate. |
| 12:06 |
gsams |
JBoyer: That's the plan, I'll try it from the start and see if that fixes it. |
| 12:06 |
gsams |
I'm sorry for all of the havoc in the process. |
| 12:06 |
berick |
bah, gsams++ |
| 12:07 |
gsams |
It's been a real learning experience, that's for sure! |
| 12:07 |
JBoyer |
It's annoying that it's giving you this much trouble. It's supposed to be fairly simple. |
| 12:07 |
JBoyer |
gsams++ # keeping with it. |
| 12:09 |
JBoyer |
(I consider that a failing of one of my changes; I don't think you've missed anything unless it's already unclear in the docs) |
| 12:20 |
JBoyer |
Oh, I just learned that PowerShell has a wget equivalent and now I'm intrigued, heh. |
| 12:20 |
* jeff |
resurrects the SIP2 flight recorder |
| 12:25 |
JBoyer |
(of course I can't use the PS downloader because the local proxy requires a login...) |
| 12:37 |
|
jihpringle joined #evergreen |
| 12:51 |
|
sandbergja joined #evergreen |
| 12:58 |
gsams |
JBoyer: Still no printers found unfortunately. |
| 12:58 |
gsams |
I'm going to go digging through the changes a little closer and see if I can find anything out of place. |
| 13:00 |
JBoyer |
I'll give everything the once-over again tonight when I get home. This time I'll try building it in Windows instead of linux on Windows and see if that helps. |
| 13:00 |
JBoyer |
I can amuse myself with it tomorrow while my car is being worked on if nothing else. |
| 13:13 |
jeff |
of course, the time to start the flight recorder is not after you've had an event requiring data from the flight recorder. |
| 13:13 |
jeff |
so it's nice that it turns out the flight recorder was running during the event. |
| 13:14 |
jeff |
and now we're just following its homing beacon to the backup repository containing the data from before it rotated out of the 48 hour cicrular buffer :-) |
| 13:34 |
JBoyer |
I can't tell which parts of that are for real and which parts are Grisham novel. This SIP message gets around but was in the wrong place at the wrong time and now it's an international incident. |
| 13:37 |
jeff |
I'm just being silly, it seems: we had a problem on Jan 15 for which we've been unable to create an isolated test case / reproduction. pcap logs of SIP2 traffic would be useful, and exist, but have rotated out of the circular dumpcap buffer. backup retention for the VM in question goes back far enough that we should be able to get the pcap data from there. |
| 13:40 |
JBoyer |
jeff++ # on the case. |
| 14:03 |
|
jamesrf joined #evergreen |
| 14:13 |
Dyrcona |
So, I'm getting this 2019-01-31 14:11:42 bh5 osrf_json_gw: [INFO:7428:osrf_app_session.c:394:154896183174287] Returning NULL from app_request_recv after timeout: open-ils.actor.patron.settings.retrieve ["redacted",1698794] in the logs for 3 users. So far, nothing looks off about their settings. |
| 14:19 |
Dyrcona |
I think it has something to do with the size of their copy templates settings. The 3 users all have web staff and xul templates that are over 200K, and they are among the largest templates settings in the database. |
| 14:50 |
Dyrcona |
Does anyone remember the message that goes in the osrf logs when a message is dropped by ejabberd because it is too large? I seem to recall there being something one could look for in the logs. |
| 15:05 |
berick |
Dyrcona: the perl warns with "Sending large message of $len bytes to...", but I don't think the C code has code supports that warning |
| 15:06 |
berick |
s/has code// |
| 15:06 |
Dyrcona |
berick: Thanks. I may also be thinking of the ejabberd message that only appears if you up the log level, IIRC. |
| 15:06 |
berick |
yeah, you do have to up the level :( |
| 15:07 |
berick |
for ejabberd |
| 15:07 |
Dyrcona |
I think there's a chunking bug with usr settings and escaped JSON. |
| 15:07 |
berick |
there is still a pending bug, iirc. |
| 15:07 |
* berick |
leaves max stanza high for now |
| 15:08 |
Dyrcona |
I thought so, but didn't check, yet. |
| 15:09 |
Dyrcona |
Our max_stanza_size is 2MB and we apparently hit it with 3 users who each have 400K of copy templates when summing up their xul and webstaff templates. |
| 15:10 |
berick |
yowza |
| 15:10 |
berick |
i wonder how many templates that is |
| 15:11 |
Dyrcona |
I didn't count individual templates, but I did run 'em through the Python json module to check that they were valid. |
| 15:11 |
Dyrcona |
After I deleted the webstaff template settings the users could work again. |
| 15:12 |
berick |
copy templates would probably benefit from having their own table, so the client can start by just fetching the labels |
| 15:12 |
berick |
and fetch individual templates on demand |
| 15:12 |
Dyrcona |
+1 |
| 15:13 |
JBoyer |
That would probably also make it easier to make sure you always have "your own" templates. The way I squeezed it in can be a little fragile when swapping users. |
| 15:17 |
Dyrcona |
For the logs: The string to look of is "XML stanza is too big" from Lp 1725317 |
| 15:17 |
pinesol |
Launchpad bug 1725317 in OpenSRF ""XML stanza is too big" still possible with chunking and bundling" [Undecided,New] https://launchpad.net/bugs/1725317 - Assigned to Galen Charlton (gmc) |
| 15:40 |
|
jvwoolf joined #evergreen |
| 15:48 |
|
khuckins joined #evergreen |
| 15:56 |
gsams |
JBoyer: Is it possible the problem isn't with the hatch installer, but with the hatch extension that is installed in Chrome? Would it take the new changes into account? |
| 15:56 |
gsams |
Oh that's totally it |
| 15:57 |
gsams |
It doesn't know to look at org.evergreen_ils.hatch.chrome.json it's still looking for the version without chrome in it because it installs the version in the web store. |
| 15:57 |
gsams |
I changed the local extension and suddenly it works. |
| 16:04 |
gsams |
So I can confirm that your fixes will work, when the chrome extension is updated to point to the right extension file in the future. |
| 16:04 |
gsams |
or if we change the extension file for the chrome extension back to what it was. |
| 16:18 |
|
yboston joined #evergreen |
| 16:36 |
berick |
gsams++ |
| 16:40 |
berick |
gsams: if you add a note to the LP entry to that affect, I'll take it from there |
| 16:42 |
gsams |
berick: you got it |
| 16:48 |
gsams |
berick: Hopefully that was enough info, but done. |
| 16:57 |
|
jvwoolf left #evergreen |
| 17:00 |
pinesol |
News from qatests: Testing Success <http://testing.evergreen-ils.org/~live> |
| 17:15 |
mmorgan |
gsams++ |
| 17:15 |
|
mmorgan left #evergreen |
| 17:47 |
|
khuckins joined #evergreen |
| 20:54 |
|
JBoyer_alt joined #evergreen |
| 21:05 |
|
JBoyer_alt_bin joined #evergreen |
| 21:14 |
|
JBoyer joined #evergreen |
