| --- Log opened Wed Jun 04 11:07:53 2008 | ||
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA | init | 11:07 | |
| wwoods | so um. | 11:14 |
|---|---|---|
| wwoods | anyone here for the QA meeting? | 11:14 |
| * DemonJester present ;) | 11:14 | |
| wwoods | DemonJester: oh hey! Haven't seen you in a while | 11:15 |
| wwoods | did anyone ask you about the F9Final testing glitch? | 11:15 |
| DemonJester | no? | 11:15 |
| wwoods | http://fedoraproject.org/wiki/QA/TestResults/Fedora9Install/FinalRelease | 11:15 |
| wwoods | says you tested a x86_64 CD install | 11:16 |
| wwoods | grr. stupid wiki | 11:16 |
| -!- cyberpear_ [n=cp@pool-71-178-37-235.washdc.east.verizon.net] has joined #fedora-meeting | 11:16 | |
| wwoods | http://fedoraproject.org/wikiold/QA/TestResults/Fedora9Install/FinalRelease | 11:16 |
| f13 | hey, we should bring andy into this. | 11:16 |
| -!- alindebe [n=alindebe@nat/redhat-us/x-759814dc33e8b111] has joined #fedora-meeting | 11:17 | |
| -!- stickster [n=nnnnnnnn@fedora/stickster] has quit [Read error: 104 (Connection reset by peer)] | 11:17 | |
| wwoods | alindebe: howdy | 11:17 |
| alindebe | wwoods: hey | 11:18 |
| wwoods | DemonJester: so yeah, at the last minute (trying to remember F9 release-time now) | 11:18 |
| -!- stickster [n=nnnnnnnn@ip72-219-206-95.dc.dc.cox.net] has joined #fedora-meeting | 11:18 | |
| wwoods | we discovered that x86_64 CD installs could not possibly work | 11:18 |
| f13 | alindebe is the QA intern we have up in Bostford for the summer, primarily working on Anaconda. | 11:18 |
| wwoods | f13: am I remembering this right | 11:18 |
| -!- tibbs [n=tibbs@fedora/tibbs] has joined #fedora-meeting | 11:19 | |
| f13 | wwoods: that is correct. Split CD installs were broken since Preview. | 11:19 |
| wwoods | it was either CD or DVD. anyway.. DemonJester, you marked the case as passed on that results page | 11:19 |
| wwoods | we've been wondering how you created the images, how you tested, etc. | 11:19 |
| f13 | nobody noticed at Preview testing nor after Preview release, and continued not ot notice it throughout the Final testing. | 11:19 |
| wwoods | so we can figure out if the test case was unclear or if you just typo'd the pass result into the wrong place or what | 11:20 |
| DemonJester | hmm.. I think I may have mistakenly put dvd in cd as I didnt do CD's for x86_64. | 11:20 |
| -!- poelcat [n=slick@fedora/poelcat] has joined #fedora-meeting | 11:20 | |
| -!- poelcat [n=slick@fedora/poelcat] has quit [Client Quit] | 11:22 | |
| wwoods | okay | 11:22 |
| -!- giallu [n=giallu@81-174-26-126.static.ngi.it] has quit [Read error: 110 (Connection timed out)] | 11:23 | |
| -!- jeff_hann [n=arares@89.40.98.185] has joined #fedora-meeting | 11:23 | |
| wwoods | well, mistakes will happen, and editing the wiki page to add results certainly messed me up a couple of times | 11:23 |
| DemonJester | sorry about my oversight with that. | 11:23 |
| wwoods | so I guess the lesson there is: we need a simple way of reporting test run results | 11:23 |
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA | Test Plans, Test Cases, Test Runs | 11:23 | |
| wwoods | so let's talk about the exciting world of test plans | 11:24 |
| DemonJester | or double check and make sure I put it in the correct spot ;) | 11:24 |
| wwoods | DemonJester: indeed | 11:24 |
| wwoods | so, we have a new wiki. And we've been using it to re-categorize our test cases | 11:24 |
| wwoods | see e.g. https://fedoraproject.org/wiki/Category:Installer_Test_Cases | 11:25 |
| -!- poelcat [n=slick@fedora/poelcat] has joined #fedora-meeting | 11:25 | |
| wwoods | jlaska and atodorov have been working on that (thanks guys) | 11:26 |
| wwoods | but you'll notice.. that page is kinda.. ugly | 11:26 |
| wwoods | the test case names aren't real easy to read | 11:26 |
| wwoods | more importantly, if we're going to expand to thousands of test cases, we probably don't want them turning up for every user search on the wiki | 11:26 |
| wwoods | so in talking with the docs team we've agreed that we should probably create a separate namespace on the wiki for test cases/plans/etc. | 11:27 |
| wwoods | this lets us go nuts with naming and categorization and stuff without worrying about colliding with the main wiki | 11:28 |
| wwoods | furthermore, mediawiki has some nice api stuff to get info about the contents of categories and the templates used in pages | 11:29 |
| wwoods | so I'm cooking up a way to do all the test reporting in a web app that pulls data out of the wiki and/or bugzilla | 11:29 |
| wwoods | so you'll be able to log in with your FAS account and start a new test run, mark tests as "passed" or "failed" or whatever, etc. | 11:30 |
| -!- kital [n=Joerg_Si@fedora/kital] has quit [Read error: 110 (Connection timed out)] | 11:30 | |
| wwoods | I'm hoping for less of this 'edit the wiki page and add this markup to turn the box green' | 11:30 |
| wwoods | and more.. choose the status, hit 'apply', and a nice dashboard of test completion, pass/fail nubmers, etc. | 11:31 |
| wwoods | basically a very lightweight version of testopia (http://www.mozilla.org/projects/testopia/) | 11:31 |
| wwoods | apparently, our engineering team is evaluating integrating testopia into our bugzilla instance in the next year or so | 11:32 |
| wwoods | but that's two Fedora releases away | 11:32 |
| wwoods | so the plan is still: write test cases on the wiki, categorize them on the wiki, etc. | 11:32 |
| wwoods | but I'm gonna need a little help with design and testing to get this testrun tracker working right | 11:33 |
| wwoods | so I might bug folks about it in the future. | 11:33 |
| wwoods | any questions about the plans there? | 11:33 |
| DemonJester | looks good to me. | 11:34 |
| wwoods | okay, cool. | 11:34 |
| wwoods | the second stage of that tool (and something that testopia already supports) | 11:35 |
| -!- ianm [n=ianm@oleg.lib.washington.edu] has joined #fedora-meeting | 11:35 | |
| wwoods | is getting results from *automated* tools | 11:35 |
| wwoods | f13 may have swayed my opinion here but I think one thing we would really, really like to have | 11:36 |
| wwoods | is automated installation testing of Rawhide | 11:36 |
| wwoods | that's going to require.. a small lab with some hardware (easy) | 11:37 |
| wwoods | and extending SNAKE a bit to handle the testing (harder) | 11:37 |
| wwoods | mostly we just need some way of signaling the machines when it's time to attempt to install rawhide | 11:37 |
| wwoods | a way to detect when they fail | 11:37 |
| wwoods | and a way to recover them when that happens | 11:37 |
| wwoods | and a method for pulling logs off the machines as they install | 11:38 |
| wwoods | the latter we've got mostly worked out in an older test system | 11:38 |
| wwoods | the rest is still incomplete. | 11:38 |
| wwoods | anyone who feels like looking into that stuff should let me know. | 11:38 |
| wwoods | I'm personally going to be focusing more on the test case management stuff for now, because that's a good entry point for getting more testers. | 11:39 |
| wwoods | and that's really what we need. | 11:39 |
| DemonJester | we are talking about internal test builds here correct? Just making sure I am on the same page | 11:39 |
| -!- jds2001 [n=jds2001@fedora/jds2001] has quit [Connection timed out] | 11:39 | |
| wwoods | DemonJester: no, we're talking about Rawhide - the development tree | 11:40 |
| wwoods | it's public. goes out to the mirrors every day. | 11:40 |
| DemonJester | glad I asked then. | 11:40 |
| wwoods | I forget what the address of the Master Mirror is, but it gets built on the public koji server | 11:41 |
| kanarip | http://download.fedora.redhat.com/pub/fedora/linux/development/ | 11:41 |
| wwoods | http://koji.fedoraproject.org/mash | 11:42 |
| wwoods | errr | 11:42 |
| wwoods | yeah. I think the koji url is where they get built.. but don't use that | 11:42 |
| wwoods | it messes with the builders. | 11:42 |
| wwoods | kanarip's got the master mirror | 11:42 |
| wwoods | anyway, having automated install testing lets us fill out the rawhide dashboard page | 11:43 |
| wwoods | e.g. http://wwoods.fedorapeople.org/rawhide.html | 11:43 |
| wwoods | so every day, rawhide will get built, and then we can check for boot images and attempt to install it | 11:43 |
| wwoods | so by the time it hits the mirrors people will know whether or not it works that day | 11:43 |
| wwoods | DemonJester: https://fedoraproject.org/wiki/Testing has some info on testing rawhide | 11:44 |
| wwoods | I need to improve it, though. | 11:45 |
| -!- jds2001 [n=jds2001@fedora/jds2001] has joined #fedora-meeting | 11:45 | |
| f13 | There is also https://fedoraproject.org/wiki/Releases/Rawhide which is in dire need of help | 11:45 |
| wwoods | f13: yeah definitely | 11:45 |
| DemonJester | yeah I already sync up with rawhide to a local mirror, I was on a different wave length though with my question. | 11:45 |
| f13 | I pointed alindebe at ^^ earlier, and for the uninitated it would be pretty useless. | 11:46 |
| wwoods | yeah, definitely | 11:46 |
| wwoods | alindebe: if you want to update that page (or just ask a lot of pointed questions about missing info) it'd help to have a fresh set of eyes on it | 11:47 |
| -!- mccann [n=jmccann@nat/redhat-us/x-d74a3ff2f00b1e68] has joined #fedora-meeting | 11:47 | |
| wwoods | I should also be talking to mmcallis (from docs) about how to go about organizing/improving our tester docs | 11:48 |
| f13 | wwoods: I was going to point alindebe at the Final release testing pages as a starting point for running today's rawhide through it's paces. | 11:48 |
| wwoods | because they're kind of in sorry shape. | 11:48 |
| wwoods | f13: you should probably use the wikiold link, since the conversion script ate all the test names | 11:49 |
| f13 | wwoods: was planning on it (: | 11:49 |
| wwoods | (why oh why can't I use sed on the wiki) | 11:49 |
| wwoods | oh, I guess I could. hmmm | 11:49 |
| f13 | hell, if i'ts got xmlrpc, lmacken can use vim with the wiki | 11:50 |
| wwoods | https://fedoraproject.org/w/api.php | 11:50 |
| wwoods | so.. kinda | 11:50 |
| wwoods | anyway, use the wikiold link until I get around to writing wikised.py | 11:51 |
| wwoods | heh | 11:51 |
| wwoods | that's all I really wanted to discuss today. any questions or other topics? | 11:52 |
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA | misc | 11:52 | |
| * wwoods takes that as a "no" | 11:53 | |
| wwoods | alright, thanks for your time, further discussion on #fedora-qa or fedora-test-list | 11:53 |
| wwoods | I'll send out my notes shortly | 11:53 |
| wwoods | you are hereby released for lunch (for those of you in US Eastern) | 11:54 |
| --- Log closed Wed Jun 04 11:54:30 2008 | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!