Time |
Nick |
Message |
00:59 |
|
book` joined #evergreen |
04:23 |
|
mtj_ joined #evergreen |
04:41 |
pinesol_green |
Incoming from qatests: Test Failure - http://testing.evergreen-ils.org/~live/test.html <http://testing.evergreen-ils.org/~live/test.html> |
07:23 |
|
mrpeters joined #evergreen |
07:34 |
|
collum joined #evergreen |
07:44 |
|
rjackson_isl joined #evergreen |
07:59 |
|
jboyer-isl joined #evergreen |
08:05 |
|
akilsdonk joined #evergreen |
08:15 |
|
Dyrcona joined #evergreen |
08:24 |
miker |
Bmagic: you'll want to use udp instead of tcp for rsyslogd. that'll solve the "clogging" problem. |
08:29 |
|
dkyle joined #evergreen |
08:40 |
|
mmorgan joined #evergreen |
08:42 |
|
maryj joined #evergreen |
08:44 |
Stompro |
Happy test writing day everyone, Woohoo |
08:44 |
jboyer-isl |
miker: In that situation, wouldn’t the logs that would be backed up just be lost? That’s obviously preferable to knocking app servers offline, but still not ideal I would think. |
08:55 |
|
ericar joined #evergreen |
08:56 |
* csharp |
confirms that PINES is using UDP to ship logs |
08:57 |
|
yboston joined #evergreen |
09:04 |
|
maryj joined #evergreen |
09:07 |
|
montgoc1 joined #evergreen |
09:08 |
ldw |
Stompro: thank you for your excitement! My West Coast brain is going to take some time to catch up :). I am going ot get a coffee, then I will be back to answer any questions. |
09:27 |
|
rfrasur joined #evergreen |
09:34 |
miker |
jboyer-isl: not necessarily. depends on buffer settings at various levels |
09:36 |
jboyer-isl |
Ah, ok. |
09:51 |
rfrasur |
Soooo, does this mean no one is volunteering to set up the festival canopy? |
09:56 |
berick |
i think you can also configure rsyslog to drop new messages when its outbound queueu fills up instead of throttling the message submitter |
09:57 |
berick |
as a last layer of defense |
10:01 |
* berick |
puts on his test-writing pants |
10:04 |
ldw |
How do we want to work on tests today? Do people want to take a bug and work on it idependently? Do we need to discuss pgTap or test writing in general? |
10:04 |
ldw |
s/idependently/independently/ |
10:10 |
|
Christineb joined #evergreen |
10:13 |
|
Shae joined #evergreen |
10:14 |
Stompro |
I think I understand what needs to be done for the evergreen.lowercase test, I just have to get things setup so I can test it. I'll let you know if I run into anything. |
10:18 |
berick |
i'm going off-script a little bit and doing a perl live test. will return to pgtap after that. |
10:18 |
jboyer-isl |
I’ll be pretty in and out as far as test writing day goes ( :( ) but I would like to recommend that tests include things expected to fail. I don’t mean tests that fail, I mean writing the tests so that they basically say “This sql/function/etc. should fail and that’s ok. If it doesn’t fail, fail this test.” |
10:20 |
ldw |
jboyer-isl: do you mean pass in data to the function that should fail it and check for the fail? |
10:21 |
jeff |
that's how i interpreted his statement. |
10:22 |
jboyer-isl |
ldw: Yes. It depends on what’s be tested of course, it doesn’t make sense for every test. The thinking is that if the test starts failing, something has broken in the function. (it’s not validating arguments correctly, it’s breaking a contract that it will never return NULL, etc.) |
10:36 |
Stompro |
Debian Wheeze has a package for pgtap 0.90 released in 2011... I wonder if that will work? I'm getting errors " ERROR: function isnt_empty(unknown, unknown) does not exist" |
10:37 |
ldw |
Stompro: you need to run CREATE EXTENSION pgtap; on your database. |
10:37 |
ldw |
I am running into issues with pgTap and Postgres 9.4. I am going to fall back to 9.1 for today. |
10:38 |
Stompro |
Yep, I've done that. |
10:39 |
ldw |
what are you passing as variables? |
10:40 |
Stompro |
I'm just running pg_prove, "pg_prove -U evergreen -d egdb -h localhost t" |
10:45 |
Stompro |
Do you install the pgtap extension for just your Evergreen db, or globally, for template1? |
10:46 |
ldw |
Stompro: your own database |
10:46 |
Stompro |
Running \dx shows "pgtap | 0.90.0 | evergreen | Unit testing for PostgreSQL |
10:46 |
Stompro |
" |
10:47 |
jboyer-isl |
Stompro: I believe things added to template1 only affect databases created after template1 is changed. Existing dbs are unaffected. |
10:47 |
Stompro |
jboyer-isl, makes sense, thanks. |
10:48 |
jboyer-isl |
The answer appears to be that pgtap .90 does not include isnt_empty, only is_empty: https://github.com/theory/pgtap/blob/v0.90.0/sql/pgtap.sql.in |
10:48 |
jboyer-isl |
That’s unfortunate. :( |
10:50 |
Stompro |
I'll switch to a debian jessie test system with a newer version, thanks for finding the answer jboyer-isl++ |
10:50 |
Dyrcona |
Stompro: You could try installing a more recent pgtap from source, just sayin'. |
10:52 |
Stompro |
I know, I just want to get going on writing the tests and not mucking with building pgtap today. |
11:01 |
Stompro |
Sigh, no debian package for pgtap in Jessie, I guess I will be building it. |
11:06 |
jeff |
looks like the pgtap package in debian is not well maintained. as such, it was removed from testing back in 2013. |
11:06 |
* berick |
grabs 1483508 |
11:06 |
dbs |
http://apt.postgresql.org/pub/repos/apt/pool/main/p/pgtap/ may be an option |
11:07 |
csharp |
I don't mean to distract from all the test-writing awesomeness, but can someone confirm this behavior on a 2.7+ system using built-in selfcheck?... |
11:07 |
* jeff |
waits for csharp's question to be about usernames/barcodes |
11:07 |
csharp |
jeff: yep ;-) |
11:07 |
dbs |
but then you're into the whole "use postgresql's apt repos instead of debian/ubuntu's" thing |
11:08 |
csharp |
with the Patron Barcode Format YAOUS set to ^[0-9]+$, when I enter a barcode or username, it appears to accept it, but doesn't accept the password after it's typed |
11:08 |
dbs |
http://wiki.postgresql.org/wiki/Apt fwiw |
11:08 |
csharp |
the web console shows that the regex works |
11:08 |
csharp |
testing <barcode#> ... barcode |
11:08 |
berick |
did we decide we can stop adding all of the \pset boilerplate at the top of these tests, since it's not needed with pg_prove ? |
11:09 |
csharp |
I see the ****** (dots) appear for the password, but Enter doesn't work |
11:10 |
csharp |
if I remove the regex from the YAOUS, it works fine |
11:10 |
Stompro |
Ok, building pgtap is about as simple as installing the package... I'm going to add a note to the qa wiki page. |
11:10 |
Stompro |
To not use the debian versions. |
11:11 |
Dyrcona |
berick: Yes, I think we did. |
11:11 |
csharp |
(but without the regex, barcodes are not accepted at all) |
11:11 |
* Dyrcona |
stopped adding them anyway. |
11:12 |
berick |
Dyrcona: cool, thanks |
11:12 |
csharp |
("accepted" = "considered to be valid user data" |
11:12 |
csharp |
) |
11:48 |
|
bmills joined #evergreen |
11:51 |
|
mrpeters joined #evergreen |
11:59 |
|
bmills joined #evergreen |
12:03 |
|
jihpringle joined #evergreen |
12:07 |
Stompro |
Is there a trick to pasting unicode characters into putty? I can view them fine, but I cannot copy them. |
12:07 |
pastebot |
"yboston" at 64.57.241.14 pasted "first pgTAP test attempt" (42 lines) at http://paste.evergreen-ils.org/24 |
12:08 |
yboston |
I was wndering if I could get some feedback on my first pgtap test. I am testing a function called public.first_agg from Open-ILS/src/sql/Pg/002.functions.aggregate.sql |
12:08 |
yboston |
LP 1483506 |
12:08 |
pinesol_green |
Launchpad bug 1483506 in Evergreen "public.first_agg needs test" (affected: 1, heat: 6) [Undecided,New] https://launchpad.net/bugs/1483506 - Assigned to Yamil (ysuarez) |
12:08 |
yboston |
what file name should I use? |
12:10 |
berick |
yboston: file name for the test? |
12:10 |
yboston |
berick: yes |
12:11 |
yboston |
also, not sure how the actual function is being used |
12:11 |
berick |
yboston: something like Open-ILS/src/sql/Pg/t/lp1483506-first-agg.pg should do |
12:11 |
yboston |
to help me test it better |
12:11 |
berick |
see other examples in Open-ILS/src/sql/Pg/t/ |
12:11 |
yboston |
OK, was not sure if I shuld put the LP number or not |
12:12 |
berick |
doesn't really matter, but may as well |
12:13 |
yboston |
the LP bug was created for todays "test day," the LP bug does not have any significant content |
12:13 |
berick |
so, first_agg() is a very simple function. pass in 2 things, it returns the first one that's not NULL. if they're both null, it returns NULL |
12:14 |
yboston |
sorry, I meant I was havign a hard time finding it used int he code. Might have grepped wrong |
12:14 |
yboston |
I am not sure if I shuld be testing with more than simple data types. not sure if I should test with passing arrays(?) |
12:14 |
berick |
hm, don't think it is used in the code. |
12:14 |
berick |
see no references to it |
12:15 |
yboston |
berick: thanks, I feel less crazy now |
12:15 |
yboston |
maybe in the past it was used, but with all the cleanign that has been done it is no longer needed |
12:16 |
berick |
maybe |
12:17 |
yboston |
berick: thanks for the feedback, I feel less crazy now. |
12:21 |
dbs |
berick: yboston: maybe no lp number in the filename, so that we can add more tests to the same file later? |
12:21 |
yboston |
dbs: makes sense |
12:22 |
yboston |
what about file name structure, shoudl I prepend the SQL file name then add the fucntion name? |
12:22 |
berick |
yeah, that makes sense. perhaps we should clean up the existing ones too |
12:22 |
berick |
re: no lp number |
12:23 |
berick |
yboston: hm, i would just do the function name |
12:23 |
yboston |
berick: good idea |
12:24 |
yboston |
can we use subfolders> |
12:24 |
yboston |
? |
12:24 |
yboston |
berick: I think first_agg gets used by calls to first() |
12:24 |
yboston |
if so, I have more ideas of how to test it |
12:25 |
berick |
yes, in fact it does |
12:25 |
yboston |
may add a test for first() too |
12:25 |
berick |
good catch |
12:27 |
Stompro |
Hmm, evergreen.lowercase doesn't seem to handle the two turkish I's correctly. |
12:28 |
yboston |
so the two ideas are soemthing like 1) 002.functions.aggregate.sql_first_agg.pg 2) first_agg.pg |
12:28 |
yboston |
3) 002.functions.aggregate/first_agg.pg (assuming subfolders are allowed inside "t" |
12:29 |
berick |
yboston: i'm sure at some point in the future, a function will move from one file to another |
12:30 |
ldw |
I like subfolders. But pg_prove needs to be run with --recurse for that to work. |
12:30 |
yboston |
berick: makes sense |
12:30 |
yboston |
ldw: good to know |
12:30 |
berick |
i mean, if we're going to use the file name, we just need to also change the test name, if we're going to include the file name in the test |
12:30 |
berick |
should a function move |
12:31 |
berick |
maybe that's very rare, not worth worrying about |
12:31 |
berick |
anyway, it popped in my head, and i blurted sans filter :) |
12:31 |
yboston |
I think we don't have to come up with a solid decision right now, I just wanted to run it by soem folks to get the cnversation started |
12:32 |
berick |
hm, i should add an aggregate test to my text_concat test as well |
12:33 |
csharp |
okay - I can confirm the same "set Patron Barcode Format setting and bork selfcheck" issue on a test server running master |
12:33 |
* csharp |
runs off to open the bug report |
12:34 |
yboston |
BTW, since pg_proves needs --recurse for subfolders to work, I will avoid using a subfolder |
12:34 |
yboston |
I look forward tpo having so many tests that we need to start using subfolders |
12:35 |
berick |
yboston++ # fixing problems before they become problems |
12:37 |
yboston |
berick for your feedback |
12:37 |
yboston |
berick++ |
12:39 |
dbs |
yboston++ |
12:39 |
dbs |
berick++ |
12:40 |
csharp |
bug 1485052 created |
12:40 |
pinesol_green |
Launchpad bug 1485052 in Evergreen "Setting Patron Barcode Format org unit setting breaks selfcheck" (affected: 1, heat: 6) [Undecided,New] https://launchpad.net/bugs/1485052 |
12:41 |
csharp |
y'all_testers++ |
12:41 |
* csharp |
has to run - see you next week |
12:58 |
bshum |
dbwells: I'm looking into recently reported https://bugs.launchpad.net/evergreen/+bug/1484989 to see if I can see what they're seeing. I asked for some more information on the matter at hand. |
12:58 |
pinesol_green |
Launchpad bug 1484989 in Evergreen "Fines are not calculating until after circulation is closed" (affected: 1, heat: 6) [Undecided,Incomplete] |
12:59 |
bshum |
But since they claim it happens after 2.8, I wonder if it's related to any reshuffling that happened in billings during development that was pushed forward. |
13:07 |
|
rlefaive joined #evergreen |
13:26 |
|
rlefaive joined #evergreen |
13:42 |
dbwells |
bshum: Thanks for the heads up. I imagine the rearrangement caused this, so I'll see if it looks like an easy fix. |
13:51 |
bshum |
dbwells: Sure thing, and Callender just added a new comment with some more specifics. |
13:52 |
bshum |
It does seem like a reordering issue, oh well :( |
13:52 |
bshum |
I guess we don't hit it cause we do fines once daily :D |
13:52 |
|
jyorio joined #evergreen |
13:53 |
dbwells |
Right, it only affects circs where there is an ungenerated fine at checkin, and the balance was zero before checkin. |
13:53 |
bshum |
Oh nevermind, I see you assigned yourself. |
13:53 |
bshum |
You're on top of it |
14:39 |
phasefx |
dbwells: unrelated, but in the same area, it looks like that having that lost/zero setting could prevent fines from being generated at all during checkin |
14:40 |
* phasefx |
hasn't actually tested that yet though |
14:41 |
dbwells |
phasefx: Thanks. I've got a fix for the first part (just writing a test), but I'll need to think more on that. The idea there is to purposely not generate fines if it is a lost checkin with currently zero fines, as that is what I understand the setting to mean. |
14:42 |
bshum |
dbwells: yeah I think that's the new setting that miker added for keeping stuff at zero. Stop gap for negative balance stuff :P |
14:43 |
phasefx |
dbwells: I did put some logging statements in testing the reported bug, and the handle_fines that gets called is the one inside if (!$dont_change_lost_zero) { |
14:44 |
* phasefx |
squints, okay, it's probably safe, since !$dont_change_lost_zero isn't just based on the setting, but looks like it considers the actual copy status, etc |
14:45 |
|
buzzy joined #evergreen |
14:45 |
dbwells |
right. It tries to just short-circuit the whole thing, and I think it's right, or at least close ;) |
14:53 |
bshum |
Okay... |
14:53 |
bshum |
So I've been asked to wipe out a bunch of collection fees for a given set of users. |
14:53 |
bshum |
The problem is, I can't seem to figure out where the money.billing links back to the user for these given entries. |
14:54 |
bshum |
They all have btype of 2, "Long overdue collections fee" |
14:54 |
bshum |
But the xact IDs don't mean anything as far as I can tell. |
14:54 |
bshum |
Or at least they dont link back to anything in action.circulation |
14:54 |
bshum |
So my wondering is, does it link anywhere else I should look for, and hopefully find something to link back to actor.usr |
14:55 |
bshum |
To verify that I'm removing the right collection fees |
14:57 |
dbwells |
bshum: maybe in money.grocery? |
14:58 |
dbwells |
bshum: if you just need to link to a user, you could probably use money.billable_xact straightaway, I think. |
14:59 |
dbwells |
The only other billable thing I know of is booking.reservation, and that probably isn't it. |
14:59 |
bshum |
Oh interesting... so the xact in money.billing likely refers to money.billable_xact.id? |
15:00 |
dbwells |
yes |
15:00 |
bshum |
That helps immensely. |
15:00 |
bshum |
Now of course, I have more users than the list i was given, but I can work with that. |
15:00 |
bshum |
dbwells++ # thanks! |
15:01 |
dbwells |
money.billable_xact is the "parent" table of action.circulation, money.grocery, and booking.reservation, so they all share a pool of ids. |
15:04 |
bshum |
After we remove or void those entries, part of me thinks I should do something to refresh the user summaries |
15:05 |
bshum |
Ah, there's a trigger on money.billing for that sort of thing, maybe... |
15:06 |
* bshum |
checks |
15:06 |
bshum |
(and tests) |
15:14 |
bshum |
Oh happy day. |
15:21 |
jboyer-isl |
Bmagic: if you are still looking into ways to relieve rsyslog congestion, look up the $ActionQueueType, $ActionQueueFileName, $SystemLogRateLimitInterval, and $SystemLogRateLimitBurst directives. |
15:22 |
jboyer-isl |
On a related note, fulling turning on that firehose for our database is yielding about 20%+ larger logs than letting the rate limiter eat thousands of what are likely 0ms queries. |
15:25 |
ldw |
jboyer-isl: I am writing negative tests, but I am using the isnt function to make sure they pass. This results in all tests passing. Does this seem reasonable to you, or is there a specific reason you had for wanting a test to fail? |
15:26 |
jboyer-isl |
ldw: that’s exactly what I meant. if you just run pg_prove it should say all tests pass. A test that passes because it’s expecting a function to fail still passes. :) |
15:26 |
ldw |
ok cool :) |
15:39 |
Stompro |
ldw, I'm looking at the pgtap test for change_db_setting , which calls alter database... but it looks like changes to alter database only take effect for new sessions. Which would be outside the testing transaction, so I don't think that is testable. What do you think? |
15:39 |
Bmagic |
jboyer-isl: thanks! for now, I am going to leave at local logging. I am afraid of bringing it all down again. Makes for an unpleasant day! |
15:39 |
ldw |
Stompro: you should enclose your tests in a BEGIN and ROLLBACK block, so the changes take effect for the duration of the test, but are not permanent. |
15:40 |
|
jlitrell joined #evergreen |
15:40 |
ldw |
Stompro: I see what you are saying |
15:40 |
ldw |
hmmm |
15:40 |
jeff |
evergreen.change_db_setting is involved in the search_path mess. |
15:40 |
Stompro |
jeff, that is the only thing it is used for that I can see. |
15:41 |
jeff |
i believe it is only used to change the db search_path, and potentially in a way that doesn't account for the fact that such changes are not immediate. i haven't dug into it further/recently. |
15:41 |
jeff |
but i remember it from my search_path digging. |
15:41 |
Stompro |
So I was trying to change the search path, and then show the search path, but if you want to change the search_path for the current session you use 'set' , not alter db. |
15:41 |
* jeff |
nods |
15:42 |
jeff |
if pg_tap supports disconnecting and reconnecting between test steps, that might be useful. if not, then this might not be a great function to test Right Now. |
15:43 |
Stompro |
But would the transaction persist across connections? |
15:43 |
jeff |
no. |
15:44 |
ldw |
could it be SELECTed from the SCHEMA instead of specifying a value to expect, specify a SQL statement? I am not sure if you can do that. |
15:44 |
Stompro |
and the change would rollback at the disconnect. So lets skip that one. |
15:45 |
Stompro |
Maybe, I'll dig and see where those values are stored. |
15:47 |
ldw |
Stompro: I am not sure if the second paramter from the tests can be a SQL statement. I might be wasting your time if you are digging through the schema. I am trying to see if I can get a SELECT statement to run as the second paramter. But, no luck so far. |
15:47 |
Stompro |
I was using results_eq Which takes a query and what the results should match. |
15:48 |
ldw |
you could run a \set after the alter to set a variable to the value of a select statement and use that value in your test. |
15:59 |
ldw |
Stompro: hmm, I cannot get \set to equal a value of a SELECT statement either. |
16:00 |
ldw |
Stompro: you can run a query as the first paramter to a pgTap test if it is enclosed in (). |
16:01 |
ldw |
So, if you know where that value is stored, you could check it that way. |
16:01 |
Stompro |
All I can find is the pg_settings virtual table, but that doesn't reflect changes made by alter database, since they only take effect for new connections after the change. |
16:08 |
ldw |
Stompro: you could try this SELECT pg_reload_conf(); |
16:09 |
ldw |
We need some no-op command for pgTap though |
16:10 |
Stompro |
pg_reload_conf cannot be run in a transaction. |
16:10 |
jeff |
i don't think that pg_reload_conf is what you want here. |
16:11 |
jeff |
``pg_reload_conf sends a SIGHUP signal to the server, causing configuration files to be reloaded by all server processes.'' |
16:11 |
Stompro |
Do we really need to test to make sure Alter Database works... can't we leave that to the postgresql guys. |
16:11 |
ldw |
Stompro: no |
16:11 |
ldw |
no we do not need to test it |
16:12 |
jeff |
i repeat my earlier: "this might not be a great function to test Right Now" :-) |
16:13 |
ldw |
I like to battle with the computer some days :/. |
16:16 |
Stompro |
jeff++, got it, I'm moving on. ldw++ thanks for the suggestions. |
16:23 |
|
dkyle left #evergreen |
16:28 |
ldw |
How should we go about including these tests in a release? |
16:31 |
Stompro |
From reading the pgtap docs, we could include the pgtap.sql file with Evergreen, and the tests could run during eg_db_config. |
16:34 |
Stompro |
Are the pgtap tests part of the current QA run? |
16:34 |
ldw |
Stompro: I am more thinking about the EG release process. Presumably we will have a number of tests attached to LP bugs, I should then target them to a release, but I am not sure which release. |
16:34 |
ldw |
Stompro: I do not know about the current QA run. |
16:36 |
Stompro |
A LP targeted test would be proving that the fix for that bug works, correct? So the test would need the fix applied before it would work? |
16:37 |
Dyrcona |
Stompro: Usually. I could also see a test that could reveal the problem before it is fixed being useful, but that could be the same thing. |
16:38 |
Dyrcona |
Most of the pgTAP tests I've written for LP bugs have been to make sure new indexes and fields exist after the upgrade script is run, for instance. |
16:40 |
ldw |
In this case though, we are writing tests for functions tha alread exist. To prevent against regression. But we still need the tests to get commited to the master branch at somepoint. Dyrcona: is a pull request enough for that? |
16:41 |
Dyrcona |
Yes it is. Someone will merge it, usually after running the tests, themselves. |
16:42 |
ldw |
Stompro: do you know how to add a pullrequest tag to LP bugs? |
16:42 |
Stompro |
Yep, I added it to the one that I completed today. |
16:42 |
ldw |
awesome. I have added one to mine as well. |
16:43 |
Stompro |
yboston, did you get yours done today? |
16:44 |
Stompro |
Just checked bug log, yes he did. |
16:47 |
ldw |
I have 6 tests listed with test-writing-day-0 and a pullrequest tag. Not a bad first showing. Hopefully, we can get more participation during the next one. Does a test writing day every three months sound too onerous? |
16:48 |
Stompro |
Not to me, that sounds just fine. |
16:54 |
dbwells |
ldw: I did write a test today for bug #1484989, if that counts :) Would have liked to do more. |
16:54 |
pinesol_green |
Launchpad bug 1484989 in Evergreen "Fines are not calculating until after circulation is closed" (affected: 1, heat: 6) [Medium,Incomplete] https://launchpad.net/bugs/1484989 |
16:54 |
ldw |
dbwells: that counts! Thank you. |
17:00 |
yboston |
lwd & Stompro there was a question earlier about running a select on the second parameter, if you mean the second param of is(), I am doing it in my test. |
17:01 |
yboston |
just had to wrap it in parens |
17:01 |
yboston |
see here... http://git.evergreen-ils.org/?p=working/Evergreen.git;a=blob;f=Open-ILS/src/sql/Pg/t/public.first.pg;h=31941b3b21f2c69d8e3125fd6e34b8f74954d437;hb=3bcef83d5438fee4355c7eaa3ea304f271cac9b5 |
17:04 |
Stompro |
Thanks yboston, have a good weekend. |
17:10 |
|
mmorgan left #evergreen |
17:14 |
pinesol_green |
Incoming from qatests: Test Success - http://testing.evergreen-ils.org/~live/test.html <http://testing.evergreen-ils.org/~live/test.html> |
17:53 |
|
vlewis joined #evergreen |
18:39 |
|
mrpeters left #evergreen |
18:40 |
|
bmills joined #evergreen |
19:18 |
|
jlitrell joined #evergreen |