| Time |
Nick |
Message |
| 00:16 |
|
sleary joined #evergreen |
| 07:18 |
|
redavis joined #evergreen |
| 08:02 |
|
BDorsey joined #evergreen |
| 08:57 |
|
Dyrcona joined #evergreen |
| 09:45 |
|
dguarrac joined #evergreen |
| 09:53 |
|
mmorgan joined #evergreen |
| 10:00 |
|
sleary joined #evergreen |
| 10:00 |
|
collum joined #evergreen |
| 10:41 |
|
smayo joined #evergreen |
| 11:13 |
|
redavis joined #evergreen |
| 11:19 |
jeffdavis |
I put a couple of LP bugs on the agenda for the next dev meeting (bug 1017990 and bug 2042158) but it turns out I won't be there to discuss them - just hoping the meeting can get a quick consensus about how to proceed with them. |
| 11:19 |
pinesol |
Launchpad bug 1017990 in Evergreen "Possible to bypass holds placement limits via direct API calls" [Medium,Confirmed] https://launchpad.net/bugs/1017990 |
| 11:19 |
pinesol |
Launchpad bug 2042158 in Evergreen "Postgres 12+ needs to have JIT disabled" [Undecided,Confirmed] https://launchpad.net/bugs/2042158 - Assigned to Galen Charlton (gmc) |
| 11:24 |
Bmagic |
jeffdavis++ |
| 11:36 |
Dyrcona |
Well, I think my opinion on both of those is unpopular... jeffdavis++ |
| 11:36 |
jeffdavis |
heh, well the consensus doesn't *have* to be quick ;) |
| 11:42 |
|
jihpringle joined #evergreen |
| 11:56 |
Bmagic |
berick: we're a week in with no issues. redis and PG15 on a production machine |
| 11:58 |
Dyrcona |
Bmagic: You disabled the JIT? |
| 11:58 |
Bmagic |
yep |
| 11:59 |
berick |
Bmagic: that's freakin awesome! thanks for letting me know. |
| 12:00 |
Bmagic |
The container boots much faster now that it doesn't have to deal with ejabberd's hostname change dance |
| 12:00 |
berick |
Bmagic: mind adding a note to the LP? https://bugs.launchpad.net/opensrf/+bug/2017941 |
| 12:00 |
pinesol |
Launchpad bug 2017941 in OpenSRF "Migrate from Ejabberd to Redis for OpenSRF Messaging" [Wishlist,Confirmed] |
| 12:00 |
Bmagic |
oh yes, I should. Will do |
| 12:00 |
berick |
Bmagic++ |
| 12:09 |
|
jvwoolf joined #evergreen |
| 12:36 |
|
adam_reid joined #evergreen |
| 12:49 |
Dyrcona |
So, we've switched SFTP off in EDI for now. We're seeing really old messages get picked up again and being reprocessed. Looks like Ingram doesn't delete old files. SFTP is apparently not doing the check for previously downloaded files. I'm will also restore the code to delete files after they're picked up that is commented out. |
| 12:50 |
Dyrcona |
"I'm will" bad editing. :) |
| 12:56 |
|
jvwoolf joined #evergreen |
| 12:58 |
Dyrcona |
I've never liked how the RemoteAccount.pm functions. Maybe I'll just refactor it.... |
| 13:07 |
Dyrcona |
mmorgan: I heard you are going to enable SFTP with EDI. I recommend holding off a bit. it's apparently grabbing files from years ago that are still on the server and processing them. |
| 13:07 |
|
sandbergja joined #evergreen |
| 13:08 |
mmorgan |
Dyrcona: Thanks for the head's up! We have been following and are holding back. |
| 13:08 |
mmorgan |
Dyrcona++ |
| 13:21 |
berick |
Dyrcona++ indeed |
| 13:27 |
Dyrcona |
Nothing like running something in production to find out how it really works. |
| 13:28 |
Dyrcona |
So, I'm thinking we should 1) delete files after retrieving and 2) improve the duplicate detection or at least make it work for SFTP. (That's without looking at the code again. I've been comparing a couple of accounts to see what's different.) |
| 13:29 |
Dyrcona |
Later, I think the RemoteAccount.pm should be refactored to connect only once per username/host pair per session/run, and by host I mean real host, because we have some cnames floating around. |
| 13:30 |
Dyrcona |
There's a bug for the latter: Lp 1836908. |
| 13:30 |
pinesol |
Launchpad bug 1836908 in Evergreen 3.11 "EDI File Transfer Needs to be Smarter" [Medium,Confirmed] https://launchpad.net/bugs/1836908 |
| 13:32 |
berick |
Dyrcona: i'd stop short of deleting the files after download since that's new behavior, but fixing the dupe detection is obv. critical. |
| 13:33 |
Dyrcona |
In the long run, I think we should delete the files, or make it an option. As far as I can tell, Ingram isn't cleaning up the outgoing directory. I have not checked other vendors, yet. |
| 13:34 |
Dyrcona |
So, I'll agree with that not being a bug fix. |
| 13:36 |
Dyrcona |
Meanwhile, I can test with local servers and messages from production. |
| 13:36 |
Dyrcona |
Wonder if the fetcher works with the debugger? |
| 13:47 |
Dyrcona |
I swear that I saw the duplicate detection code last week, but can I find it right now? Nope. |
| 13:48 |
Dyrcona |
It's probably in OpenILS::Application::Acq::EDI->process_retrieval |
| 13:50 |
Dyrcona |
Close. It's in retrieve_core.... |
| 13:58 |
Dyrcona |
So, it must be caused by a difference in the output of ls_sftp viz ls_ftp. |
| 13:59 |
Dyrcona |
Think I'll have to implement a wrapper around remote account to see the difference. |
| 13:59 |
|
sleary joined #evergreen |
| 14:04 |
Dyrcona |
Interesting: Running the fetcher as-is the first time, it finds the 2 files and downloads them once. I get expected output: EDI: outgoing/45974.EPR already retrieved. Skipping at /usr/local/share/perl/5.34.0/OpenILS/Application/Acq/EDI.pm line 159. |
| 14:04 |
Dyrcona |
I noticed that last week. |
| 14:04 |
Dyrcona |
So duplicate detection appears to be working. |
| 14:05 |
Dyrcona |
A second fetcher run: Files retrieved: 0 |
| 14:05 |
Dyrcona |
And those duplicate file messages for each account. |
| 14:10 |
Dyrcona |
Ugh. I wish acq worked on 3.7. |
| 14:14 |
Dyrcona |
Files retrieved: 9 <- Is totally wrong because there are only 4 files to retrieve. |
| 14:14 |
|
jvwoolf joined #evergreen |
| 14:21 |
Dyrcona |
That said, a second run retrieves 0 files and it looks like one of the invoices actually covers multiple purchase orders. |
| 14:22 |
Dyrcona |
So, it looks like we may have an issue that Eg version or distro-related. |
| 14:25 |
Dyrcona |
Which would be quicker: upgrade 3.7 to the same branch that works or fire up a new vm? |
| 14:25 |
Dyrcona |
In the long run, probably the latter. |
| 14:32 |
|
jvwoolf joined #evergreen |
| 14:39 |
Dyrcona |
And, the network doesn't want to work after I configure just like it's buddy, but with a different IP. Maybe something else is using the IP I chose? |
| 14:41 |
* Dyrcona |
starts over. I think It got the wrong net device from the host. |
| 14:50 |
Dyrcona |
That's better. It was using the default NAT when I wanted the bridge. |
| 15:12 |
|
jihpringle joined #evergreen |
| 15:26 |
|
jvwoolf joined #evergreen |
| 16:03 |
|
sandbergja joined #evergreen |
| 16:11 |
|
jvwoolf joined #evergreen |
| 16:30 |
Dyrcona |
OK. I cannot reproduce the duplicate retrieval in any of my 3 test environments with recent files. |
| 16:31 |
Dyrcona |
This requires further investigation. |
| 16:52 |
Dyrcona |
Well, that's that fo today. |
| 16:59 |
|
Guest21 joined #evergreen |
| 16:59 |
|
jvwoolf left #evergreen |
| 17:00 |
|
mmorgan left #evergreen |
| 18:19 |
|
jihpringle joined #evergreen |
| 23:43 |
|
kmlussier joined #evergreen |
| 23:43 |
kmlussier |
@praise late-night bug wranglers |
| 23:43 |
* pinesol |
<cameron_voice>late-night bug wranglers you're kmlussier's HERO</cameron_voice> |
| 23:43 |
kmlussier |
It's true |
| 23:53 |
|
kmlussier left #evergreen |
