Evergreen ILS Website

IRC log for #evergreen, 2015-08-14

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat

All times shown according to the server's local time.

Time Nick Message
00:59 book` joined #evergreen
04:23 mtj_ joined #evergreen
04:41 pinesol_green Incoming from qatests: Test Failure - http://testing.evergreen-ils.org/~live/test.html <http://testing.evergreen-ils.org/~live/test.html>
07:23 mrpeters joined #evergreen
07:34 collum joined #evergreen
07:44 rjackson_isl joined #evergreen
07:59 jboyer-isl joined #evergreen
08:05 akilsdonk joined #evergreen
08:15 Dyrcona joined #evergreen
08:24 miker Bmagic:  you'll want to use udp instead of tcp for rsyslogd.  that'll solve the "clogging" problem.
08:29 dkyle joined #evergreen
08:40 mmorgan joined #evergreen
08:42 maryj joined #evergreen
08:44 Stompro Happy test writing day everyone, Woohoo
08:44 jboyer-isl miker: In that situation, wouldn’t the logs that would be backed up just be lost? That’s obviously preferable to knocking app servers offline, but still not ideal I would think.
08:55 ericar joined #evergreen
08:56 * csharp confirms that PINES is using UDP to ship logs
08:57 yboston joined #evergreen
09:04 maryj joined #evergreen
09:07 montgoc1 joined #evergreen
09:08 ldw Stompro: thank you for your excitement! My West Coast brain is going to take some time to catch up :).  I am going ot get a coffee, then I will be back to answer any questions.
09:27 rfrasur joined #evergreen
09:34 miker jboyer-isl: not necessarily. depends on buffer settings at various levels
09:36 jboyer-isl Ah, ok.
09:51 rfrasur Soooo, does this mean no one is volunteering to set up the festival canopy?
09:56 berick i think you can also configure rsyslog to drop new messages when its outbound queueu fills up instead of throttling the message submitter
09:57 berick as a last layer of defense
10:01 * berick puts on his test-writing pants
10:04 ldw How do we want to work on tests today? Do people want to take a bug and work on it idependently?  Do we need to discuss pgTap or test writing in general?
10:04 ldw s/idependently/independently/
10:10 Christineb joined #evergreen
10:13 Shae joined #evergreen
10:14 Stompro I think I understand what needs to be done for the evergreen.lowercase test, I just have to get things setup so I can test it.  I'll let you know if I run into anything.
10:18 berick i'm going off-script a little bit and doing a perl live test.  will return to pgtap after that.
10:18 jboyer-isl I’ll be pretty in and out as far as test writing day goes ( :(  ) but I would like to recommend that tests include things expected to fail. I don’t mean tests that fail, I mean writing the tests so that they basically say “This sql/function/etc. should fail and that’s ok. If it doesn’t fail, fail this test.”
10:20 ldw jboyer-isl: do you mean pass in data to the function that should fail it and check for the fail?
10:21 jeff that's how i interpreted his statement.
10:22 jboyer-isl ldw: Yes. It depends on what’s be tested of course, it doesn’t make sense for every test. The thinking is that if the test starts failing, something has broken in the function. (it’s not validating arguments correctly, it’s breaking a contract that it will never return NULL, etc.)
10:36 Stompro Debian Wheeze has a package for pgtap 0.90 released in 2011... I wonder if that will work?  I'm getting errors " ERROR:  function isnt_empty(unknown, unknown) does not exist"
10:37 ldw Stompro: you need to run CREATE EXTENSION pgtap; on your database.
10:37 ldw I am running into issues with pgTap and Postgres 9.4.  I am going to fall back to 9.1 for today.
10:38 Stompro Yep, I've done that.
10:39 ldw what are you passing as variables?
10:40 Stompro I'm just running pg_prove,  "pg_prove -U evergreen -d egdb -h localhost t"
10:45 Stompro Do you install the pgtap extension for just your Evergreen db, or globally, for template1?
10:46 ldw Stompro: your own database
10:46 Stompro Running \dx shows "pgtap     | 0.90.0  | evergreen  | Unit testing for PostgreSQL
10:46 Stompro "
10:47 jboyer-isl Stompro: I believe things added to template1 only affect databases created after template1 is changed. Existing dbs are unaffected.
10:47 Stompro jboyer-isl, makes sense, thanks.
10:48 jboyer-isl The answer appears to be that pgtap .90 does not include isnt_empty, only is_empty: https://github.com/theory/pgtap​/blob/v0.90.0/sql/pgtap.sql.in
10:48 jboyer-isl That’s unfortunate. :(
10:50 Stompro I'll switch to a debian jessie test system with a newer version, thanks for finding the answer jboyer-isl++
10:50 Dyrcona Stompro: You could try installing a more recent pgtap from source, just sayin'.
10:52 Stompro I know, I just want to get going on writing the tests and not mucking with building pgtap today.
11:01 Stompro Sigh, no debian package for pgtap in Jessie, I guess I will be building it.
11:06 jeff looks like the pgtap package in debian is not well maintained. as such, it was removed from testing back in 2013.
11:06 * berick grabs 1483508
11:06 dbs http://apt.postgresql.org/pub​/repos/apt/pool/main/p/pgtap/ may be an option
11:07 csharp I don't mean to distract from all the test-writing awesomeness, but can someone confirm this behavior on a 2.7+ system using built-in selfcheck?...
11:07 * jeff waits for csharp's question to be about usernames/barcodes
11:07 csharp jeff: yep ;-)
11:07 dbs but then you're into the whole "use postgresql's apt repos instead of debian/ubuntu's" thing
11:08 csharp with the Patron Barcode Format YAOUS set to ^[0-9]+$, when I enter a barcode or username, it appears to accept it, but doesn't accept the password after it's typed
11:08 dbs http://wiki.postgresql.org/wiki/Apt fwiw
11:08 csharp the web console shows that the regex works
11:08 csharp testing <barcode#> ... barcode
11:08 berick did we decide we can stop adding all of the \pset boilerplate at the top of these tests, since it's not needed with pg_prove ?
11:09 csharp I see the ****** (dots) appear for the password, but Enter doesn't work
11:10 csharp if I remove the regex from the YAOUS, it works fine
11:10 Stompro Ok, building pgtap is about as simple as installing the package...  I'm going to add a note to the qa wiki page.
11:10 Stompro To not use the debian versions.
11:11 Dyrcona berick: Yes, I think we did.
11:11 csharp (but without the regex, barcodes are not accepted at all)
11:11 * Dyrcona stopped adding them anyway.
11:12 berick Dyrcona: cool, thanks
11:12 csharp ("accepted" = "considered to be valid user data"
11:12 csharp )
11:48 bmills joined #evergreen
11:51 mrpeters joined #evergreen
11:59 bmills joined #evergreen
12:03 jihpringle joined #evergreen
12:07 Stompro Is there a trick to pasting unicode characters into putty?  I can view them fine, but I cannot copy them.
12:07 pastebot "yboston" at 64.57.241.14 pasted "first pgTAP test attempt" (42 lines) at http://paste.evergreen-ils.org/24
12:08 yboston I was wndering if I could get some feedback on my first pgtap test. I am testing a function called public.first_agg from Open-ILS/src/sql/Pg/002.functions.aggregate.sql
12:08 yboston LP 1483506
12:08 pinesol_green Launchpad bug 1483506 in Evergreen "public.first_agg needs test" (affected: 1, heat: 6) [Undecided,New] https://launchpad.net/bugs/1483506 - Assigned to Yamil (ysuarez)
12:08 yboston what file name should I use?
12:10 berick yboston: file name for the test?
12:10 yboston berick: yes
12:11 yboston also, not sure how the actual function is being used
12:11 berick yboston: something like Open-ILS/src/sql/Pg/t/lp1483506-first-agg.pg should do
12:11 yboston to help me test it better
12:11 berick see other examples in Open-ILS/src/sql/Pg/t/
12:11 yboston OK, was not sure if I shuld put the LP number or not
12:12 berick doesn't really matter, but may as well
12:13 yboston the LP bug was created for todays "test day," the LP bug does not have any significant content
12:13 berick so, first_agg() is a very simple function.  pass in 2 things, it returns the first one that's not NULL.  if they're both null, it returns NULL
12:14 yboston sorry, I meant I was havign a hard time finding it used int he code. Might have grepped wrong
12:14 yboston I am not sure if I shuld be testing with more than simple data types. not sure if I should test with passing arrays(?)
12:14 berick hm, don't think it is used in the code.
12:14 berick see no references to it
12:15 yboston berick: thanks, I feel less crazy now
12:15 yboston maybe in the past it was used, but with all the cleanign that has been done it is no longer needed
12:16 berick maybe
12:17 yboston berick: thanks for the feedback, I feel less crazy now.
12:21 dbs berick: yboston: maybe no lp number in the filename, so that we can add more tests to the same file later?
12:21 yboston dbs: makes sense
12:22 yboston what about file name structure, shoudl I prepend the SQL file name then add the fucntion name?
12:22 berick yeah, that makes sense.  perhaps we should clean up the existing ones too
12:22 berick re: no lp number
12:23 berick yboston: hm, i would just do the function name
12:23 yboston berick: good idea
12:24 yboston can we use subfolders>
12:24 yboston ?
12:24 yboston berick: I think first_agg gets used by calls to first()
12:24 yboston if so, I have more ideas of how to test it
12:25 berick yes, in fact it does
12:25 yboston may add a test for first() too
12:25 berick good catch
12:27 Stompro Hmm, evergreen.lowercase doesn't seem to handle the two turkish I's correctly.
12:28 yboston so the two ideas are soemthing like  1) 002.functions.aggregate.sql_first_agg.pg  2) first_agg.pg
12:28 yboston 3) 002.functions.aggregate/first_agg.pg (assuming subfolders are allowed inside "t"
12:29 berick yboston: i'm sure at some point in the future, a function will move from one file to another
12:30 ldw I like subfolders.  But pg_prove needs to be run with --recurse for that to work.
12:30 yboston berick: makes sense
12:30 yboston ldw: good to know
12:30 berick i mean, if we're going to use the file name, we just need to also change the test name, if we're going to include the file name in the test
12:30 berick should a function move
12:31 berick maybe that's very rare, not worth worrying about
12:31 berick anyway, it popped in my head, and i blurted sans filter :)
12:31 yboston I think we don't have to come up with a solid decision right now, I just wanted to run it by soem folks to get the cnversation started
12:32 berick hm, i should add an aggregate test to my text_concat test as well
12:33 csharp okay - I can confirm the same "set Patron Barcode Format setting and bork selfcheck" issue on a test server running master
12:33 * csharp runs off to open the bug report
12:34 yboston BTW, since pg_proves needs --recurse for subfolders to work, I will avoid using a subfolder
12:34 yboston I look forward tpo having so many tests that we need to start using subfolders
12:35 berick yboston++ # fixing problems before they become problems
12:37 yboston berick for your feedback
12:37 yboston berick++
12:39 dbs yboston++
12:39 dbs berick++
12:40 csharp bug 1485052 created
12:40 pinesol_green Launchpad bug 1485052 in Evergreen "Setting Patron Barcode Format org unit setting breaks selfcheck" (affected: 1, heat: 6) [Undecided,New] https://launchpad.net/bugs/1485052
12:41 csharp y'all_testers++
12:41 * csharp has to run - see you next week
12:58 bshum dbwells: I'm looking into recently reported https://bugs.launchpad.net/evergreen/+bug/1484989 to see if I can see what they're seeing.  I asked for some more information on the matter at hand.
12:58 pinesol_green Launchpad bug 1484989 in Evergreen "Fines are not calculating until after circulation is closed" (affected: 1, heat: 6) [Undecided,Incomplete]
12:59 bshum But since they claim it happens after 2.8, I wonder if it's related to any reshuffling that happened in billings during development that was pushed forward.
13:07 rlefaive joined #evergreen
13:26 rlefaive joined #evergreen
13:42 dbwells bshum: Thanks for the heads up.  I imagine the rearrangement caused this, so I'll see if it looks like an easy fix.
13:51 bshum dbwells: Sure thing, and Callender just added a new comment with some more specifics.
13:52 bshum It does seem like a reordering issue, oh well :(
13:52 bshum I guess we don't hit it cause we do fines once daily :D
13:52 jyorio joined #evergreen
13:53 dbwells Right, it only affects circs where there is an ungenerated fine at checkin, and the balance was zero before checkin.
13:53 bshum Oh nevermind, I see you assigned yourself.
13:53 bshum You're on top of it
14:39 phasefx dbwells: unrelated, but in the same area, it looks like that having that lost/zero setting could prevent fines from being generated at all during checkin
14:40 * phasefx hasn't actually tested that yet though
14:41 dbwells phasefx: Thanks.  I've got a fix for the first part (just writing a test), but I'll need to think more on that.  The idea there is to purposely not generate fines if it is a lost checkin with currently zero fines, as that is what I understand the setting to mean.
14:42 bshum dbwells: yeah I think that's the new setting that miker added for keeping stuff at zero.  Stop gap for negative balance stuff :P
14:43 phasefx dbwells: I did put some logging statements in testing the reported bug, and the handle_fines that gets called is the one inside if (!$dont_change_lost_zero) {
14:44 * phasefx squints, okay, it's probably safe, since !$dont_change_lost_zero isn't just based on the setting, but looks like it considers the actual copy status, etc
14:45 buzzy joined #evergreen
14:45 dbwells right.  It tries to just short-circuit the whole thing, and I think it's right, or at least close ;)
14:53 bshum Okay...
14:53 bshum So I've been asked to wipe out a bunch of collection fees for a given set of users.
14:53 bshum The problem is, I can't seem to figure out where the money.billing links back to the user for these given entries.
14:54 bshum They all have btype of 2, "Long overdue collections fee"
14:54 bshum But the xact IDs don't mean anything as far as I can tell.
14:54 bshum Or at least they dont link back to anything in action.circulation
14:54 bshum So my wondering is, does it link anywhere else I should look for, and hopefully find something to link back to actor.usr
14:55 bshum To verify that I'm removing the right collection fees
14:57 dbwells bshum: maybe in money.grocery?
14:58 dbwells bshum: if you just need to link to a user, you could probably use money.billable_xact straightaway, I think.
14:59 dbwells The only other billable thing I know of is booking.reservation, and that probably isn't it.
14:59 bshum Oh interesting... so the xact in money.billing likely refers to money.billable_xact.id?
15:00 dbwells yes
15:00 bshum That helps immensely.
15:00 bshum Now of course, I have more users than the list i was given, but I can work with that.
15:00 bshum dbwells++ # thanks!
15:01 dbwells money.billable_xact is the "parent" table of action.circulation, money.grocery, and booking.reservation, so they all share a pool of ids.
15:04 bshum After we remove or void those entries, part of me thinks I should do something to refresh the user summaries
15:05 bshum Ah, there's a trigger on money.billing for that sort of thing, maybe...
15:06 * bshum checks
15:06 bshum (and tests)
15:14 bshum Oh happy day.
15:21 jboyer-isl Bmagic: if you are still looking into ways to relieve rsyslog congestion, look up the $ActionQueueType, $ActionQueueFileName, $SystemLogRateLimitInterval, and $SystemLogRateLimitBurst directives.
15:22 jboyer-isl On a related note, fulling turning on that firehose for our database is yielding about 20%+ larger logs than letting the rate limiter eat thousands of what are likely 0ms queries.
15:25 ldw jboyer-isl: I am writing negative tests, but I am using the isnt function to make sure they pass.  This results in all tests passing.  Does this seem reasonable to you, or is there a specific reason you had for wanting a test to fail?
15:26 jboyer-isl ldw: that’s exactly what I meant. if you just run pg_prove it should say all tests pass. A test that passes because it’s expecting a function to fail still passes. :)
15:26 ldw ok cool :)
15:39 Stompro ldw, I'm looking at the pgtap test for change_db_setting , which calls alter database... but it looks like changes to alter database only take effect for new sessions.  Which would be outside the testing transaction, so I don't think that is testable.   What do you think?
15:39 Bmagic jboyer-isl: thanks! for now, I am going to leave at local logging. I am afraid of bringing it all down again. Makes for an unpleasant day!
15:39 ldw Stompro: you should enclose your tests in a BEGIN and ROLLBACK block, so the changes take effect for the duration of the test, but are not permanent.
15:40 jlitrell joined #evergreen
15:40 ldw Stompro: I see what you are saying
15:40 ldw hmmm
15:40 jeff evergreen.change_db_setting is involved in the search_path mess.
15:40 Stompro jeff, that is the only thing it is used for that I can see.
15:41 jeff i believe it is only used to change the db search_path, and potentially in a way that doesn't account for the fact that such changes are not immediate. i haven't dug into it further/recently.
15:41 jeff but i remember it from my search_path digging.
15:41 Stompro So I was trying to change the search path, and then show the search path, but if you want to change the search_path for the current session you use 'set' , not alter db.
15:41 * jeff nods
15:42 jeff if pg_tap supports disconnecting and reconnecting between test steps, that might be useful. if not, then this might not be a great function to test Right Now.
15:43 Stompro But would the transaction persist across connections?
15:43 jeff no.
15:44 ldw could it be SELECTed from the SCHEMA instead of specifying a value to expect, specify a SQL statement?  I am not sure if you can do that.
15:44 Stompro and the change would rollback at the disconnect.  So lets skip that one.
15:45 Stompro Maybe, I'll dig and see where those values are stored.
15:47 ldw Stompro: I am not sure if the second paramter from the tests can be a SQL statement.  I might be wasting your time if you are digging through the schema.  I am trying to see if I can get a SELECT statement to run as the second paramter.  But, no luck so far.
15:47 Stompro I was using results_eq  Which takes a query and what the results should match.
15:48 ldw you could run a \set after the alter to set a variable to the value of a select statement and use that value in your test.
15:59 ldw Stompro: hmm, I cannot get \set to equal a value of a SELECT statement either.
16:00 ldw Stompro: you can run a query as the first paramter to a pgTap test if it is enclosed in ().
16:01 ldw So, if you know where that value is stored, you could check it that way.
16:01 Stompro All I can find is the pg_settings virtual table, but that doesn't reflect changes made by alter database, since they only take effect for new connections after the change.
16:08 ldw Stompro: you could try this SELECT pg_reload_conf();
16:09 ldw We need some no-op command for pgTap though
16:10 Stompro pg_reload_conf cannot be run in a transaction.
16:10 jeff i don't think that pg_reload_conf is what you want here.
16:11 jeff ``pg_reload_conf sends a SIGHUP signal to the server, causing configuration files to be reloaded by all server processes.''
16:11 Stompro Do we really need to test to make sure Alter Database works... can't we leave that to the postgresql guys.
16:11 ldw Stompro: no
16:11 ldw no we do not need to test it
16:12 jeff i repeat my earlier: "this might not be a great function to test Right Now" :-)
16:13 ldw I like to battle with the computer some days :/.
16:16 Stompro jeff++, got it, I'm moving on.  ldw++ thanks for the suggestions.
16:23 dkyle left #evergreen
16:28 ldw How should we go about including these tests in a release?
16:31 Stompro From reading the pgtap docs, we could include the pgtap.sql file with Evergreen, and the tests could run during eg_db_config.
16:34 Stompro Are the pgtap tests part of the current QA run?
16:34 ldw Stompro: I am more thinking about the EG release process.  Presumably we will have a number of tests attached to LP bugs, I should then target them to a release, but I am not sure which release.
16:34 ldw Stompro: I do not know about the current QA run.
16:36 Stompro A LP targeted test would be proving that the fix for that bug works, correct?  So the test would need the fix applied before it would work?
16:37 Dyrcona Stompro: Usually. I could also see a test that could reveal the problem before it is fixed being useful, but that could be the same thing.
16:38 Dyrcona Most of the pgTAP tests I've written for LP bugs have been to make sure new indexes and fields exist after the upgrade script is run, for instance.
16:40 ldw In this case though, we are writing tests for functions tha alread exist.  To prevent against regression.  But we still need the tests to get commited to the master branch at somepoint.  Dyrcona: is a pull request enough for that?
16:41 Dyrcona Yes it is. Someone will merge it, usually after running the tests, themselves.
16:42 ldw Stompro: do you know how to add a pullrequest tag to LP bugs?
16:42 Stompro Yep, I added it to the one that I completed today.
16:42 ldw awesome. I have added one to mine as well.
16:43 Stompro yboston, did you get yours done today?
16:44 Stompro Just checked bug log, yes he did.
16:47 ldw I have 6 tests listed with test-writing-day-0 and a pullrequest tag.  Not a bad first showing.  Hopefully, we can get more participation during the next one.  Does a test writing day every three months sound too onerous?
16:48 Stompro Not to me, that sounds just fine.
16:54 dbwells ldw: I did write a test today for bug #1484989, if that counts :)  Would have liked to do more.
16:54 pinesol_green Launchpad bug 1484989 in Evergreen "Fines are not calculating until after circulation is closed" (affected: 1, heat: 6) [Medium,Incomplete] https://launchpad.net/bugs/1484989
16:54 ldw dbwells: that counts! Thank you.
17:00 yboston lwd & Stompro there was a question earlier about running a select on the second parameter, if you mean the second param of is(), I am doing it in my test.
17:01 yboston just had to wrap it in parens
17:01 yboston see here... http://git.evergreen-ils.org/?p=working/Evergre​en.git;a=blob;f=Open-ILS/src/sql/Pg/t/public.fi​rst.pg;h=31941b3b21f2c69d8e3125fd6e34b8f74954d4​37;hb=3bcef83d5438fee4355c7eaa3ea304f271cac9b5
17:04 Stompro Thanks yboston, have a good weekend.
17:10 mmorgan left #evergreen
17:14 pinesol_green Incoming from qatests: Test Success - http://testing.evergreen-ils.org/~live/test.html <http://testing.evergreen-ils.org/~live/test.html>
17:53 vlewis joined #evergreen
18:39 mrpeters left #evergreen
18:40 bmills joined #evergreen
19:18 jlitrell joined #evergreen

| Channels | #evergreen index | Today | | Search | Google Search | Plain-Text | summary | Join Webchat