| --- Log opened Wed May 21 11:02:32 2008 | ||
| wwoods | oh nice, I finally saved my log settings properly | 11:02 |
|---|---|---|
| wwoods | fedora-qa-20080521.log opened. whee | 11:03 |
| jds2001 | hehe i have this logged going back probably as far as you need. | 11:03 |
| wwoods | what I really wish we had was a meeting-logbot | 11:03 |
| wwoods | that I could give it 'start' and 'stop' commands | 11:03 |
| wwoods | and it'd irc2html the messages between those two things and put 'em up on some website | 11:04 |
| f13 | wwoods: put up a "bounty" | 11:04 |
| f13 | Debian has such a bot... | 11:04 |
| wwoods | a bounty, eh | 11:04 |
| ianweller | hmm. zodbot can probably be told to do that, there's a ChannelLogger plugin for supybot. just fyi | 11:04 |
| f13 | ianweller: good to know! | 11:04 |
| wwoods | huh. nice. | 11:05 |
| wwoods | that'd be a really useful thing for this channel. | 11:05 |
| * ricky is still waiting on packaging supybot for Fedoar :-) | 11:05 | |
| ricky | **Fedora | 11:05 |
| wwoods | /msg zodbot start meeting-id [password].. or I guess it could just require a known, identified user | 11:06 |
| wwoods | anyway | 11:06 |
| -!- kital [n=Joerg_Si@fedora/kital] has quit [Remote closed the connection] | 11:06 | |
| wwoods | so, yes, QA meeting go | 11:06 |
| -!- sahak [n=sahak@pool-71-184-182-102.bstnma.east.verizon.net] has joined #fedora-meeting | 11:06 | |
| wwoods | we're in the part of the cycle where we can actually do planning and development | 11:06 |
| wwoods | and we are soon to have an intern (yay) | 11:06 |
| f13 | YAY! | 11:07 |
| wwoods | so we should talk tools for a bit | 11:07 |
| jds2001 | is rawhide actually installable for them to do something? I haven't tried in the last few days | 11:07 |
| wwoods | well here's the thing - I'm not so sure I just want to throw her at installing rawhide every day | 11:07 |
| wwoods | (I haven't tried either, to be honest) | 11:08 |
| f13 | jds2001: I haven't tried (: | 11:08 |
| f13 | I was meaning to today | 11:08 |
| wwoods | I've been doing a looooot of F8->F9 upgrades | 11:08 |
| wwoods | for preupgrade stuff | 11:09 |
| wwoods | really want to get that done-ish so I can move on for a while | 11:09 |
| * jds2001 noticed that preupgrade does not use his local mirror | 11:09 | |
| wwoods | jds2001: it does if you edit releases.list | 11:09 |
| jds2001 | so it's not using failovermethod=priority | 11:09 |
| wwoods | or if you set up mirrormanager | 11:09 |
| wwoods | it's using whatever yum's default is | 11:09 |
| * jds2001 has mirrormanager setup | 11:10 | |
| wwoods | which should be failovermethod=priority in F8 | 11:10 |
| wwoods | check /etc/yum.conf, 'cuz it works fine here | 11:10 |
| jds2001 | hmm, probably OT for here, but I'm seeing otherwise :) | 11:10 |
| * jds2001 can try again tonight | 11:10 | |
| wwoods | perhaps you have an out-of-date mirrorlist.txt in your preupgrade cache dir | 11:10 |
| wwoods | anyway | 11:10 |
| -!- tibbs_ is now known as tibbs|h | 11:12 | |
| wwoods | my main priority for the F10 cycle is expanding the QA team and making it easier for new people to do useful testing | 11:12 |
| -!- mether [n=ask@fedora/mether] has joined #fedora-meeting | 11:12 | |
| wwoods | so we need stuff like: better testing docs. stickster has introduced me to one of the docs guys who's looking to help with that | 11:13 |
| wwoods | we also need stuff like: tools for tracking testing (and testers) | 11:13 |
| stickster | wwoods++ !!!1!! | 11:14 |
| wwoods | further, I think we need a QA group in FAS | 11:14 |
| * f13 throws rocks at stickster | 11:14 | |
| f13 | wwoods: or to take over the existing one? | 11:14 |
| wwoods | that works too | 11:14 |
| jds2001 | just remove everybody and turn the reigns over to wwoods? | 11:15 |
| wwoods | you can make me an admin for 'qa' and I can boot everyone from the group, I guess? | 11:15 |
| * viking-ice likes then plan about "hostile" take over | 11:15 | |
| wwoods | that too | 11:15 |
| -!- SmootherFrOgZ [n=r@fedora/SmootherFrOgZ] has quit ["Leaving"] | 11:15 | |
| wwoods | but I want new contributors to have, y'know. a few choices about things to do, docs on how to do it, a list of people to ask when they have questions | 11:16 |
| wwoods | and I want *us* to have a way to see what people are up to | 11:16 |
| wwoods | updates-testing (bodhi) and triage (bugzilla) already have pretty good docs and tools | 11:16 |
| wwoods | and we can get good metroids (heh, metrics) out of them | 11:16 |
| jds2001 | bugzilla metroids are harder - but I think that poelcat had some ideas around that. | 11:17 |
| wwoods | as for release testing - all we've got is the wiki matrix. which we can hand-count, but yuck | 11:17 |
| jds2001 | any aspiring web developers to develop an app for it? | 11:17 |
| wwoods | jds2001: cool - we should talk in more detail about that at some point, but I totally trust you guys to come up with some good ideas | 11:18 |
| wwoods | well that's the thing - our intern has some background in web development | 11:18 |
| wwoods | so getting something like a test case/plan tracker, or bugzilla metroid tracker, or similar web-based tester tools | 11:18 |
| * viking-ice get's more and more curious about this mysterious intern... | 11:18 | |
| wwoods | is a distinct possiblity | 11:18 |
| f13 | wwoods is now an admin of the qa group. | 11:18 |
| * wwoods flexes | 11:18 | |
| wwoods | other proposed web tools.. the ever-popular rawhide dashboard | 11:19 |
| f13 | Honestly we do have software coming down teh pike for test case management and test result management | 11:19 |
| * viking-ice throws the dices to see if wwoods gonna smite him.. | 11:19 | |
| wwoods | f13: oh? | 11:19 |
| f13 | so I would think using the intern for something else would be better. | 11:19 |
| f13 | wwoods: that's stage 1 of bill peck's work | 11:19 |
| wwoods | but for manual test cases etc? | 11:19 |
| wwoods | I mean the Testify2 team has been talking about this for a year | 11:20 |
| f13 | wwoods: from what I understand it, the way it's supposed tow ork is that you create a test case that /is/ the manual way of doing the test | 11:20 |
| f13 | wwoods: then people can do the test and register with the system that they've done said test / steps | 11:20 |
| * f13 wonders why it is I know more about this than the guy in QA... | 11:20 | |
| wwoods | heh. because you sit near him? | 11:21 |
| f13 | anywho, there's supposed to be something working or to the point of being worked on by FUDCon BOS | 11:22 |
| * jds2001 hears pins drop | 11:22 | |
| wwoods | that'd be cool | 11:22 |
| wwoods | so, okay, assuming that bpeck is going to descend from the heavens with a perfect test tracker | 11:22 |
| f13 | I would still urge you to seek bpeck out and have a conversation with him bout this and the intern stuff. | 11:23 |
| wwoods | definitely | 11:23 |
| wwoods | anyway: what would you suggest that we work on? | 11:23 |
| wwoods | that being, y'know, the intern and whoever else has time | 11:23 |
| wwoods | as it is now, QA gets two real windows for development: summer and winter | 11:24 |
| wwoods | so. summer dev cycle priorities? | 11:24 |
| f13 | test cases would be good | 11:24 |
| f13 | or are you talking about infrastructure level stuff? | 11:24 |
| wwoods | infrastructure-level stuff | 11:25 |
| f13 | I know that I could use a lot of tree validation tests and some form of automated rawhide validation each night | 11:25 |
| wwoods | test cases are OK, a tool to put test cases in is even better | 11:25 |
| f13 | nod | 11:25 |
| wwoods | nightly rawhide runs means extending SNAKE | 11:25 |
| -!- cweyl [n=cweyl@c-24-130-175-238.hsd1.ca.comcast.net] has joined #fedora-meeting | 11:25 | |
| wwoods | which is something jlaska might be thinking about already | 11:26 |
| wwoods | IIRC he's been talking to bpeck about using SNAKE to get machines installed for the next-gen testing system | 11:26 |
| wwoods | so it's an active area of development with a good amount of appeal internally, which means we will probably get support from RHEL QE | 11:27 |
| wwoods | so is that the worst part of the test cycle? | 11:28 |
| jlaska | I've heard several suggestions from devleopment mgmt on needs for a QA intern in the fedora space that are visible in both fedora and RHEL | 11:28 |
| wwoods | making sure rawhide works and finding out when it does? | 11:28 |
| wwoods | I'd sure like to have some stats on how many days we have installable rawhide and how many days it's broken | 11:29 |
| jds2001 | i would *love* to see snake work via PXE | 11:29 |
| jds2001 | not just from an installed system | 11:29 |
| wwoods | oh yeah | 11:29 |
| wwoods | IIRC there were some changes in the livecd tools to make, basically, huge initrd images | 11:29 |
| wwoods | it'd be perfect to just use a known-good kernel and an initrd that just runs snake | 11:29 |
| jds2001 | yeah, i noticed a livecd-iso-to-pxeboot command | 11:30 |
| wwoods | so if we can figure out how to generate a super-minimal snake livecd | 11:30 |
| wwoods | then we are golden | 11:31 |
| wwoods | in fact.. snake livecd is a good idea by itself | 11:31 |
| f13 | wwoods: nightly testing is definitely a sore point right now. I'd also love to be able to poop out a full split iso tree and turn loose an automated test suite upon it | 11:31 |
| -!- giallu [n=giallu@81-174-9-190.dynamic.ngi.it] has joined #fedora-meeting | 11:31 | |
| jds2001 | f13: wow, that'd be painful :) | 11:31 |
| f13 | mmmm fibre | 11:32 |
| wwoods | better than pooping out Everything | 11:32 |
| wwoods | okay, I'm getting the sense that automated installation testing is the Big Dream | 11:33 |
| -!- poelcat [n=slick@fedora/poelcat] has joined #fedora-meeting | 11:33 | |
| wwoods | it's kind of a prereq for automated functional testing, too, so it's pretty important | 11:33 |
| wwoods | So we'll assume bpeck has the testcase / testplan tracking stuff covered | 11:35 |
| f13 | hopefully, and maybe he could use some help with it | 11:35 |
| f13 | I also think we need to focus on testing more than just the install path | 11:36 |
| f13 | working with the SIGs to create test cases to validate their area of responsiblities | 11:36 |
| jds2001 | f13++ | 11:36 |
| poelcat | f13: i think that should be part of more specific release criteria | 11:36 |
| jds2001 | right now, even though it's on the release criteria, i dont think we test that apps survive a basic click-through. | 11:36 |
| poelcat | and would consider help leading such a session at fudcon | 11:37 |
| wwoods | right | 11:37 |
| wwoods | the thing I'm worried about is that.. it leads to enormous growth in test cases | 11:37 |
| wwoods | and tracking test execution is still kind of a PITA with the wiki | 11:38 |
| jds2001 | well, if the SIGs are responsible for having someone in the QA subproject, then we're golden. | 11:38 |
| wwoods | definitely | 11:38 |
| f13 | right | 11:38 |
| wwoods | oh, and I'm going to nominate myself for FESCo so I have a vote on all proposed features | 11:38 |
| f13 | Unless the SIG is offering somebody up to do the testing, those tests are not going to be blockers | 11:38 |
| wwoods | and I'm gonna vote against anything that doesn't have a good test plan | 11:38 |
| -!- mether [n=ask@fedora/mether] has quit [Read error: 113 (No route to host)] | 11:38 | |
| * jds2001 votes for wwoods | 11:38 | |
| jds2001 | not that it looks like there's gonna be much voting at this point :/ | 11:39 |
| * jds2001 was thinking of nominating himself too, but was gonna wait til the last minute. | 11:39 | |
| -!- spoleeba [n=one@fedora/Jef] has joined #fedora-meeting | 11:39 | |
| wwoods | I mean, obviously QA as a team will offer support in writing (and executing) test plans | 11:40 |
| wwoods | so it shouldn't be too heavy a requirement | 11:40 |
| wwoods | but yeah | 11:40 |
| * viking-ice suggest "hostile" takeover since it worked so well last time ( wwoods is now an admin of the qa group ) | 11:40 | |
| wwoods | ALSO I want test plans to be attached to, y'know, components | 11:40 |
| wwoods | so when there's a new update for a package, bodhi can show all the test cases for that package | 11:41 |
| poelcat | wwoods: can you provide guidance that I can add to feature policy about what is considered sufficient | 11:41 |
| poelcat | ? | 11:41 |
| poelcat | i get a lot of push back on "how hard" it is to fill out feature pages :) | 11:41 |
| poelcat | so i'm sure there will be more if we tighten up test plan section | 11:42 |
| wwoods | there's a testplan template and plenty of examples | 11:42 |
| f13 | basically we'd like a way to verify that the Feature is working as expected, or not working as expected | 11:42 |
| * jds2001 also thinks that folks that offer that pushback are full of crap. | 11:42 | |
| wwoods | err, test *case* rather. http://fedoraproject.org/wiki/QA/TestCases/TestCaseTemplate | 11:42 |
| jds2001 | We're not asking for a doctoral thesis there. | 11:42 |
| wwoods | We'll find a good example or two | 11:43 |
| wwoods | and link people to the testcasetemplate | 11:43 |
| * jds2001 goes to lunch | 11:43 | |
| jlaska | so the thinking is, that when a feature is proposed, it would be "a good thing" to have some guidance as to steps to validate functionality | 11:44 |
| wwoods | poelcat: is that sufficient? | 11:44 |
| jlaska | are we expecting the feature author to write the testcase(s)? | 11:44 |
| wwoods | right - it's a simple request. Feature owners, please tell QA how to test your feature | 11:44 |
| poelcat | wwoods: can you give me links to the "good example or two" ? | 11:44 |
| wwoods | jlaska: yes, but we'll certainly provide assistance and guidance | 11:45 |
| jlaska | wwoods: I'm guessing they'll likely want to provide the assistance and guidance, and we focus on what we do well (the test cases) part. | 11:45 |
| wwoods | poelcat: offhand no - there's probably a couple good 'uns in the installation test plan | 11:46 |
| wwoods | http://fedoraproject.org/wiki/QA/TestCases/InstallDualbootMacOSX is nicely verbose | 11:46 |
| jlaska | I don't know if I'm comfortable asking development to write the test cases ... perhaps propose test cases, or provide input that influences a test case. | 11:46 |
| wwoods | http://fedoraproject.org/wiki/QA/TestCases/PartitioningNoSwap is more to the point | 11:46 |
| wwoods | jlaska: okay, ask that they *propose* a test case, and QA will expand/cleanup until it's acceptable | 11:47 |
| f13 | if not actual test case, psuedo test case. | 11:47 |
| -!- rdieter is now known as rdieter_away | 11:47 | |
| wwoods | well I mean, check the PartitioningNoSwap case | 11:47 |
| -!- rdieter [n=rdieter@pcp088900pcs.unl.edu] has joined #fedora-meeting | 11:47 | |
| jlaska | wwoods: I think that's fair ... they use the same philosophy for patch review for their code. We should employ the same mechanism | 11:47 |
| wwoods | that's a sufficient case, and it doesn't take real dang long to write something like that | 11:48 |
| jlaska | many features often require fairly extensive test plans ... another case where having tighter involvement between both groups is required to produce a usable outcome | 11:48 |
| wwoods | right | 11:48 |
| wwoods | larger features will probably be driven by an internal team or a SIG | 11:49 |
| -!- rdieter [n=rdieter@pcp088900pcs.unl.edu] has quit [Remote closed the connection] | 11:49 | |
| wwoods | which should already have QA reps | 11:49 |
| wwoods | so they should have some good guidance on doing the right thing | 11:49 |
| wwoods | so let's talk about the QA group a bit - any prereqs for membership? | 11:49 |
| -!- rdieter [n=rdieter@pcp088900pcs.unl.edu] has joined #fedora-meeting | 11:50 | |
| f13 | FAS and CLA | 11:50 |
| wwoods | at the moment it doesn't grant you and rights | 11:50 |
| wwoods | err, any rights | 11:50 |
| f13 | the FAS is rather implicit | 11:50 |
| f13 | but CLA I'd like so we cna be sure to re-use any submitted test cases in RHEL products | 11:50 |
| wwoods | is group membership needed for bodhi commentary, or just a FAS account? | 11:50 |
| viking-ice | Directions for turning "debugging" where to look what to strace etc.. and which component ( if not plain obvious ) to file against in bgz, will need to be on the test case page ( new testers will need this stuff ).. | 11:50 |
| f13 | bodhi commentary can be done anonymously | 11:51 |
| wwoods | right, but anonymous doesn't count | 11:51 |
| f13 | wwoods: I don't know the answer to that. | 11:51 |
| f13 | lmacken: ping? | 11:51 |
| -!- rdieter [n=rdieter@pcp088900pcs.unl.edu] has left #fedora-meeting ["Konversation terminated!"] | 11:51 | |
| wwoods | "real QA" should be done by identified users - plus we want to keep metroids on karma given by QA team members | 11:51 |
| viking-ice | +1 | 11:51 |
| jlaska | wwoods: QA group bit ... what's the driver behind that? Are there problems/concerns in the current process? | 11:52 |
| jlaska | heh ... s/Are/What are/ | 11:52 |
| wwoods | yeah, helps identify the team members and keep the team more team-y | 11:53 |
| wwoods | in the future we'll use it to give access to QA tools | 11:53 |
| viking-ice | So will this extend to overall QA or just testing.. | 11:53 |
| wwoods | like bpeck's stuff I'd assume | 11:53 |
| wwoods | overall QA - writing test plans, f'rinstance | 11:53 |
| wwoods | or if we ever start doing things like unit testing in the package database | 11:54 |
| wwoods | so, yeah, mostly the driver is recognizing and rewarding contributors to QA | 11:55 |
| wwoods | anyway we're very near the end of our hour | 11:56 |
| wwoods | any last-minute questions / discussion topics? | 11:56 |
| viking-ice | Well it should give the maintainers/developers more time to focus on their work if they can hand over all the testing to QA | 11:57 |
| wwoods | we don't really have the capacity for that right now though | 11:57 |
| viking-ice | and just post I need testing on this and this and this .. | 11:57 |
| wwoods | which is why team-building is important - to expand the team and build (and measure) capacity | 11:57 |
| viking-ice | true | 11:59 |
| * f13 foods. | 11:59 | |
| wwoods | yeah, I think meeting time is over, and food time is now | 11:59 |
| smooge | wwoods, good eating | 11:59 |
| wwoods | thanks for your time, folks.. further discussion can be moved to #fedora-qa or fedora-test-list | 12:00 |
| wwoods | oh! one note | 12:00 |
| wwoods | http://partner-bugzilla.redhat.com/ has a beta of the Bugzilla 3.2 code we'll be using soon | 12:00 |
| wwoods | it doesn't send mail or anything, so mess around to your heart's content | 12:00 |
| wwoods | *especially* if you've got any bugzilla-specific code, test it against that instance | 12:00 |
| wwoods | and make sure everything works OK | 12:00 |
| -!- stahnma [n=stahnma@fedora/stahnma] has joined #fedora-meeting | 12:01 | |
| wwoods | thanks again! | 12:01 |
| --- Log closed Wed May 21 12:01:28 2008 | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!