fedora-qa-20080604

--- Log opened Wed Jun 04 11:07:53 2008
-!- wwoods changed the topic of #fedora-meeting to: Fedora QA | init11:07
wwoodsso um.11:14
wwoodsanyone here for the QA meeting?11:14
* DemonJester present ;)11:14
wwoodsDemonJester: oh hey! Haven't seen you in a while11:15
wwoodsdid anyone ask you about the F9Final testing glitch?11:15
DemonJesterno?11:15
wwoodshttp://fedoraproject.org/wiki/QA/TestResults/Fedora9Install/FinalRelease11:15
wwoodssays you tested a x86_64 CD install11:16
wwoodsgrr. stupid wiki11:16
-!- cyberpear_ [n=cp@pool-71-178-37-235.washdc.east.verizon.net] has joined #fedora-meeting11:16
wwoodshttp://fedoraproject.org/wikiold/QA/TestResults/Fedora9Install/FinalRelease11:16
f13hey, we should bring andy into this.11:16
-!- alindebe [n=alindebe@nat/redhat-us/x-759814dc33e8b111] has joined #fedora-meeting11:17
-!- stickster [n=nnnnnnnn@fedora/stickster] has quit [Read error: 104 (Connection reset by peer)]11:17
wwoodsalindebe: howdy11:17
alindebewwoods: hey11:18
wwoodsDemonJester: so yeah, at the last minute (trying to remember F9 release-time now)11:18
-!- stickster [n=nnnnnnnn@ip72-219-206-95.dc.dc.cox.net] has joined #fedora-meeting11:18
wwoodswe discovered that x86_64 CD installs could not possibly work11:18
f13alindebe is the QA intern we have up in Bostford for the summer, primarily working on Anaconda.11:18
wwoodsf13: am I remembering this right11:18
-!- tibbs [n=tibbs@fedora/tibbs] has joined #fedora-meeting11:19
f13wwoods: that is correct.  Split CD installs were broken since Preview.11:19
wwoodsit was either CD or DVD. anyway.. DemonJester, you marked the case as passed on that results page11:19
wwoodswe've been wondering how you created the images, how you tested, etc.11:19
f13nobody noticed at Preview testing nor after Preview release, and continued not ot notice it throughout the Final testing.11:19
wwoodsso we can figure out if the test case was unclear or if you just typo'd the pass result into the wrong place or what11:20
DemonJesterhmm.. I think I may have mistakenly put dvd in cd as I didnt do CD's for x86_64.11:20
-!- poelcat [n=slick@fedora/poelcat] has joined #fedora-meeting11:20
-!- poelcat [n=slick@fedora/poelcat] has quit [Client Quit]11:22
wwoodsokay11:22
-!- giallu [n=giallu@81-174-26-126.static.ngi.it] has quit [Read error: 110 (Connection timed out)]11:23
-!- jeff_hann [n=arares@89.40.98.185] has joined #fedora-meeting11:23
wwoodswell, mistakes will happen, and editing the wiki page to add results certainly messed me up a couple of times11:23
DemonJestersorry about my oversight with that.11:23
wwoodsso I guess the lesson there is: we need a simple way of reporting test run results11:23
-!- wwoods changed the topic of #fedora-meeting to: Fedora QA | Test Plans, Test Cases, Test Runs11:23
wwoodsso let's talk about the exciting world of test plans11:24
DemonJesteror double check and make sure I put it in the correct spot ;)11:24
wwoodsDemonJester: indeed11:24
wwoodsso, we have a new wiki. And we've been using it to re-categorize our test cases11:24
wwoodssee e.g. https://fedoraproject.org/wiki/Category:Installer_Test_Cases11:25
-!- poelcat [n=slick@fedora/poelcat] has joined #fedora-meeting11:25
wwoodsjlaska and atodorov have been working on that (thanks guys)11:26
wwoodsbut you'll notice.. that page is kinda.. ugly11:26
wwoodsthe test case names aren't real easy to read11:26
wwoodsmore importantly, if we're going to expand to thousands of test cases, we probably don't want them turning up for every user search on the wiki11:26
wwoodsso in talking with the docs team we've agreed that we should probably create a separate namespace on the wiki for test cases/plans/etc.11:27
wwoodsthis lets us go nuts with naming and categorization and stuff without worrying about colliding with the main wiki11:28
wwoodsfurthermore, mediawiki has some nice api stuff to get info about the contents of categories and the templates used in pages11:29
wwoodsso I'm cooking up a way to do all the test reporting in a web app that pulls data out of the wiki and/or bugzilla11:29
wwoodsso you'll be able to log in with your FAS account and start a new test run, mark tests as "passed" or "failed" or whatever, etc.11:30
-!- kital [n=Joerg_Si@fedora/kital] has quit [Read error: 110 (Connection timed out)]11:30
wwoodsI'm hoping for less of this 'edit the wiki page and add this markup to turn the box green'11:30
wwoodsand more.. choose the status, hit 'apply', and a nice dashboard of test completion, pass/fail nubmers, etc.11:31
wwoodsbasically a very lightweight version of testopia (http://www.mozilla.org/projects/testopia/)11:31
wwoodsapparently, our engineering team is evaluating integrating testopia into our bugzilla instance in the next year or so11:32
wwoodsbut that's two Fedora releases away11:32
wwoodsso the plan is still: write test cases on the wiki, categorize them on the wiki, etc.11:32
wwoodsbut I'm gonna need a little help with design and testing to get this testrun tracker working right11:33
wwoodsso I might bug folks about it in the future. 11:33
wwoodsany questions about the plans there?11:33
DemonJesterlooks good to me.11:34
wwoodsokay, cool.11:34
wwoodsthe second stage of that tool (and something that testopia already supports)11:35
-!- ianm [n=ianm@oleg.lib.washington.edu] has joined #fedora-meeting11:35
wwoodsis getting results from *automated* tools11:35
wwoodsf13 may have swayed my opinion here but I think one thing we would really, really like to have11:36
wwoodsis automated installation testing of Rawhide11:36
wwoodsthat's going to require.. a small lab with some hardware (easy)11:37
wwoodsand extending SNAKE a bit to handle the testing (harder)11:37
wwoodsmostly we just need some way of signaling the machines when it's time to attempt to install rawhide11:37
wwoodsa way to detect when they fail11:37
wwoodsand a way to recover them when that happens11:37
wwoodsand a method for pulling logs off the machines as they install11:38
wwoodsthe latter we've got mostly worked out in an older test system11:38
wwoodsthe rest is still incomplete.11:38
wwoodsanyone who feels like looking into that stuff should let me know.11:38
wwoodsI'm personally going to be focusing more on the test case management stuff for now, because that's a good entry point for getting more testers.11:39
wwoodsand that's really what we need.11:39
DemonJesterwe are talking about internal test builds here correct? Just making sure I am on the same page11:39
-!- jds2001 [n=jds2001@fedora/jds2001] has quit [Connection timed out]11:39
wwoodsDemonJester: no, we're talking about Rawhide - the development tree11:40
wwoodsit's public. goes out to the mirrors every day.11:40
DemonJesterglad I asked then.11:40
wwoodsI forget what the address of the Master Mirror is, but it gets built on the public koji server11:41
kanariphttp://download.fedora.redhat.com/pub/fedora/linux/development/11:41
wwoodshttp://koji.fedoraproject.org/mash11:42
wwoodserrr11:42
wwoodsyeah. I think the koji url is where they get built.. but don't use that11:42
wwoodsit messes with the builders.11:42
wwoodskanarip's got the master mirror11:42
wwoodsanyway, having automated install testing lets us fill out the rawhide dashboard page11:43
wwoodse.g. http://wwoods.fedorapeople.org/rawhide.html11:43
wwoodsso every day, rawhide will get built, and then we can check for boot images and attempt to install it11:43
wwoodsso by the time it hits the mirrors people will know whether or not it works that day11:43
wwoodsDemonJester: https://fedoraproject.org/wiki/Testing has some info on testing rawhide11:44
wwoodsI need to improve it, though.11:45
-!- jds2001 [n=jds2001@fedora/jds2001] has joined #fedora-meeting11:45
f13There is also https://fedoraproject.org/wiki/Releases/Rawhide which is in dire need of help11:45
wwoodsf13: yeah definitely11:45
DemonJesteryeah I already sync up with rawhide to a local mirror, I was on a different wave length though with my question.11:45
f13I pointed alindebe at ^^ earlier, and for the uninitated it would be pretty useless.11:46
wwoodsyeah, definitely11:46
wwoodsalindebe: if you want to update that page (or just ask a lot of pointed questions about missing info) it'd help to have a fresh set of eyes on it11:47
-!- mccann [n=jmccann@nat/redhat-us/x-d74a3ff2f00b1e68] has joined #fedora-meeting11:47
wwoodsI should also be talking to mmcallis (from docs) about how to go about organizing/improving our tester docs11:48
f13wwoods: I was going to point alindebe at the Final release testing pages as a starting point for running today's rawhide through it's paces.11:48
wwoodsbecause they're kind of in sorry shape.11:48
wwoodsf13: you should probably use the wikiold link, since the conversion script ate all the test names11:49
f13wwoods: was planning on it (:11:49
wwoods(why oh why can't I use sed on the wiki)11:49
wwoodsoh, I guess I could. hmmm11:49
f13hell, if i'ts got xmlrpc, lmacken can use vim with the wiki11:50
wwoodshttps://fedoraproject.org/w/api.php11:50
wwoodsso.. kinda11:50
wwoodsanyway, use the wikiold link until I get around to writing wikised.py11:51
wwoodsheh11:51
wwoodsthat's all I really wanted to discuss today. any questions or other topics?11:52
-!- wwoods changed the topic of #fedora-meeting to: Fedora QA | misc11:52
* wwoods takes that as a "no"11:53
wwoodsalright, thanks for your time, further discussion on #fedora-qa or fedora-test-list11:53
wwoodsI'll send out my notes shortly11:53
wwoodsyou are hereby released for lunch (for those of you in US Eastern)11:54
--- Log closed Wed Jun 04 11:54:30 2008

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!