| --- Log opened Wed Dec 03 11:02:07 2008 | ||
| * jlaska | 11:02 | |
| -!- kiilerix [n=mk@cpe.ge-1-1-0-787.0x50a05c02.alb2nqu1.customer.tele.dk] has joined #fedora-meeting | 11:02 | |
| viking_ice_ | * | 11:02 |
|---|---|---|
| wwoods | and anyone else who's here for the QA meeting say hi for the logs | 11:02 |
| linuxguru | hi all ;) | 11:02 |
| wwoods | agenda for this week: | 11:03 |
| wwoods | 1) quick review of the F10 release | 11:03 |
| wwoods | 2) requirements for wikified test plans | 11:03 |
| wwoods | 3) misc. | 11:03 |
| -!- poelcat [n=slick@fedora/poelcat] has joined #fedora-meeting | 11:03 | |
| -!- JSchmitt [n=s4504kr@fedora/JSchmitt] has joined #fedora-meeting | 11:03 | |
| wwoods | and, yeah, I know, I should have sent a reminder to fedora-test-list earlier this week | 11:04 |
| -!- Evil_Sonar_Chick is now known as Sonar_Gal | 11:04 | |
| wwoods | I'll do that before the next meeting. | 11:04 |
| * f13 | 11:04 | |
| wwoods | poelcat: hiya | 11:04 |
| wwoods | okay, so let's start with.. | 11:04 |
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA Meeting | F10 Release Review | 11:05 | |
| wwoods | You probably already noticed, but: F10 is out! Whee! | 11:05 |
| * jds2001 here | 11:05 | |
| * sgtd | 11:05 | |
| -!- spoleeba [n=one@fedora/Jef] has joined #fedora-meeting | 11:05 | |
| wwoods | so. Other than the *general* QA problems/needs - more automation, better test plans, better bug reporting, etc. | 11:05 |
| wwoods | were there any *specific* problems (or good things) with the F10 release? | 11:06 |
| jlaska | well, I enjoyed the test days, and I'd like to kick off more of them for F11 features | 11:06 |
| wwoods | My Intel-video / KVM / bluetooth test machine died right around Preview, which hampered my personal testing | 11:06 |
| f13 | Our tree has some bad inconsistancies, and bugs in it | 11:06 |
| linuxguru | oh well F10 is a good release besides just system-config-network pissed a few people around i know including me yes! | 11:07 |
| wwoods | but I'm trying to get some new hardware for that | 11:07 |
| f13 | the most embarrassing is conflicts within the tree | 11:07 |
| wwoods | f13: in F10? | 11:07 |
| f13 | and broken deps in the Fedora tree and on DVD/CDs | 11:07 |
| wwoods | oh man, yuck. | 11:07 |
| -!- laubersm [n=susan@fedora/laubersm] has joined #fedora-meeting | 11:07 | |
| f13 | I missed testing for this, and everybody else did too because I had to stage the beast directly on the master mirror where nobody else could get to it | 11:08 |
| wwoods | well, conflicts within the Everything tree should be expected, right? | 11:08 |
| f13 | I ran verify-tree, but forgot to look at things verify-tree didn't check for. | 11:08 |
| f13 | wwoods: no | 11:08 |
| f13 | wwoods: at any point in time, the entire Fedora package set must be conflict free | 11:08 |
| wwoods | oh, *file* conflicts? | 11:08 |
| f13 | yeah | 11:08 |
| f13 | explicit conflicts are not allowed either | 11:09 |
| wwoods | ah OK. I thought you were saying packages could not use Conflicts: | 11:09 |
| viking_ice_ | yeah I would like to take the test days to the next level + kicking in early feature testing so we can nail them down for final.. | 11:09 |
| f13 | no current Fedora package can conflict in any other way with any other current Fedora package. | 11:09 |
| wwoods | which I thought was kind of sensible for some packages (e.g. two mutually-exclusive programs that do the same thing?) | 11:09 |
| wwoods | hm, OK | 11:09 |
| f13 | packages can use Conflicts, to express in compatability with older, or newer packages | 11:09 |
| f13 | but at no time should those Conflicts trigger on current offerings for a release. | 11:10 |
| f13 | no F8 package can conflict with a current F8 package, etc | 11:10 |
| wwoods | so let's add two tests to the release-test-checklist: 1) No Conflicts, and 2) No broken deps | 11:10 |
| f13 | yep | 11:10 |
| wwoods | the Everything install was supposed to be the de-facto test for those | 11:10 |
| f13 | (arguably they could go in verify-tree) | 11:10 |
| wwoods | but it's not really feasible anymore | 11:10 |
| jlaska | f13: can seth's conflicts checking code can be used to assist here? | 11:10 |
| f13 | yeah, this is the kind of thing we can check outside of an actual install. | 11:10 |
| f13 | one thing the everything install does give us though, noisy install logs | 11:11 |
| f13 | I don't think any check was done to make sure there were no errors in install.log files | 11:11 |
| f13 | jlaska: yes | 11:11 |
| f13 | The reasoning behind no conflicts is that the error case there is to punt to the user with a "UR DOING IT WRONG" type message, and they're left to figure out wtf they're supposed to do | 11:12 |
| f13 | or worse, composes fail | 11:12 |
| wwoods | okay. ACTION: new tool (or extend verify-tree) to check for 1) broken deps, 2) conflicts | 11:12 |
| f13 | (like people making custom live images) | 11:12 |
| wwoods | ACTION: add tests for broken deps and conflicts to release test matrix | 11:12 |
| wwoods | anyone opposed to those? | 11:12 |
| * wwoods doubts it, moving on - but speak up if you disagree | 11:13 | |
| wwoods | As for Everything - it's basically impossible to test %packages * in the Rawhide/Everything repo | 11:14 |
| -!- Netsplit niven.freenode.net <-> irc.freenode.net quits: rsc, poelcat, kanarip, mbonnet_, sm|test-box, killefiz | 11:14 | |
| wwoods | so we need a Fedora tree composed to test that.. and sometimes that's tricky to get to | 11:14 |
| f13 | wwoods: why is it impossible? | 11:14 |
| f13 | the amount of time it takes? | 11:14 |
| jlaska | time and kernels it installs | 11:14 |
| jds2001 | wwoods: i did it once or twice by checking everything in anaconda (which is obviously not the same, but caught some conflicts) | 11:14 |
| f13 | kernels is a trivial thing | 11:15 |
| f13 | especially since you can adjust that in %post | 11:15 |
| jlaska | jds2001: that's the best manual method | 11:15 |
| wwoods | AFAIK if we explicitly disallow conflicts it should be feasible | 11:15 |
| wwoods | but conflicts were the reason that Everything installs were mostly ignored | 11:15 |
| f13 | rhm, yeah those are bugs that should be fixed | 11:15 |
| wwoods | they take fooooreeeever, but that's "hard" not "impossible" | 11:15 |
| jds2001 | i thought there was always something on the ReleaseCriteria page saying "there must be no conflicts" | 11:15 |
| jlaska | installation isn't required to validate whether a yum repo contains packages that have deps conflicts or whose files conflict | 11:15 |
| f13 | it means we're just leaving pitfalls for our users to fall into | 11:16 |
| -!- kanarip`_ [n=kanarip@vito.kanarip.com] has joined #fedora-meeting | 11:16 | |
| -!- Netsplit over, joins: poelcat, rsc, killefiz, sm|test-box, mbonnet_ | 11:16 | |
| mcepl | wwoods: sorry, for jumping in the middle of something else, but I came late -- I had on computer exim installed against my will (again) -- do we have "no exim installed" in our QA tests? | 11:16 |
| f13 | jlaska: that's correct. But it is somewhat necessary to ensure every %post is clean | 11:16 |
| f13 | (or %pre/%post) | 11:16 |
| wwoods | mcepl: ha. that's a comps issue, I think | 11:16 |
| jlaska | f13: how do you mean clean? | 11:16 |
| -!- Nirmal [n=npathak@nat/redhat-in/x-40ea43f63fca96c0] has joined #fedora-meeting | 11:16 | |
| f13 | jlaska: quiet. Not littering install.log with errors. | 11:17 |
| jds2001 | jlaska: no output | 11:17 |
| -!- kanarip`_ is now known as kanarip | 11:17 | |
| mcepl | (unfortunately, I was not able to reproduce it and not having yum logs, I haven't persuaded jk about it) | 11:17 |
| wwoods | ooh, that's another good bikeshed/honeypot change I can propose - No MTA By Default | 11:17 |
| f13 | mcepl: wwoods: it's not an comps issue, it's a pebcak issue | 11:17 |
| jlaska | f13: right ... but that's a different test that checking for file conflicts | 11:17 |
| f13 | jlaska: correct. | 11:17 |
| mcepl | f13: sorry, what's pebcak? | 11:17 |
| f13 | mcepl: getting to exim is as simple as unchecking sendmail from the base group when you install. | 11:17 |
| f13 | mcepl: Problem Exists Between Chair and Keyboard. | 11:17 |
| -!- JSchmitt [n=s4504kr@fedora/JSchmitt] has quit [Remote closed the connection] | 11:18 | |
| -!- Nirmal [n=npathak@nat/redhat-in/x-40ea43f63fca96c0] has quit [Client Quit] | 11:18 | |
| -!- Nirmal [n=npathak@nat/redhat-in/x-d901686e2cb27c27] has joined #fedora-meeting | 11:18 | |
| wwoods | right, so the underlying problem is that something requires an MTA | 11:18 |
| mcepl | no, I had installed postfix and I hadn't do any selection whatsoever -- just doing yum upgrade | 11:18 |
| wwoods | and if you don't have one selected, exim wins | 11:18 |
| wwoods | does postfix not have the proper provides? | 11:18 |
| mcepl | I DID HAVE POSTFIX INSTALLED | 11:18 |
| -!- Nirmal [n=npathak@nat/redhat-in/x-d901686e2cb27c27] has quit [Client Quit] | 11:19 | |
| f13 | mcepl: then I demand yum logs (: | 11:19 |
| jds2001 | so there was a bug against cronie that it needs local delivery and doesn't require an MTA | 11:19 |
| mcepl | f13: I know, and I haven't have them, so I loose; just that that "exim wins" should be checked against in QA tests. | 11:19 |
| f13 | in what way? | 11:20 |
| f13 | "exim wins" is 100% valid, if you don't give the resolver any other hints as to what you prefer. | 11:20 |
| jds2001 | and we came up in the bug with a virtual provides for local delivery, and have an esmtp package that just does local delivery | 11:20 |
| wwoods | anyway the exim thing is certainly a bug worth following up on.. but it's not a specific problem in the test/QA process for F10 | 11:21 |
| wwoods | so let's save that for later | 11:21 |
| mcepl | sure, shutting up | 11:21 |
| -!- J5 [n=quintice@c-76-24-17-105.hsd1.ma.comcast.net] has joined #fedora-meeting | 11:21 | |
| jds2001 | https://bugzilla.redhat.com/show_bug.cgi?id=472710 | 11:21 |
| buggbot | Bug 472710: medium, medium, ---, mmaslano@redhat.com, ASSIGNED, fix cronie MTA requirements | 11:21 |
| wwoods | any other problems in the F10 test cycles that we could address in F11? | 11:22 |
| wwoods | I've got one thing - we're going to try to make the freezes freezier, and make the features have better specifications and test plans | 11:22 |
| jds2001 | just the lack of feature specs, but I'm running point on that for F11. | 11:22 |
| wwoods | jds2001: hooray, and thank you for that. | 11:23 |
| jlaska | go jds2001! :) | 11:23 |
| wwoods | To explain a bit further (IIRC) we're advising FESCo that all feature plans need a Specification (i.e. a well-formed Scope section) by Alpha | 11:23 |
| f13 | I think the biggest thing I'm focusing on for QA in F11 is getting some darned infrastructure in place for reactive testing | 11:23 |
| f13 | IE react to a cvs checking, react to a build, react to an update request, react to a tree compose | 11:24 |
| wwoods | that describes exactly what the feature is supposed to do - what pieces it needs to be deemed complete and operational | 11:24 |
| wwoods | and that Spec should be used to write a good Test Plan by Beta | 11:24 |
| wwoods | that describes how we, the testers, can actually check to see if the feature meets its specification. | 11:24 |
| -!- judazz [n=gronslet@cm-84.208.182.112.getinternet.no] has joined #fedora-meeting | 11:24 | |
| wwoods | (jds2001: did I get that right?) | 11:24 |
| jds2001 | wwoods: yep | 11:25 |
| jds2001 | and one that doesn't assume i use gizmo X every day. | 11:25 |
| wwoods | f13: ah - this falls into the broad "more automation!" category | 11:25 |
| jds2001 | (use it, see if it works is not a valid test plan IOW) | 11:25 |
| wwoods | unless there's some specific problem from the F10 cycle that you want to address with this? | 11:26 |
| -!- openpercept [n=openperc@unaffiliated/openpercept] has joined #fedora-meeting | 11:26 | |
| f13 | wwoods: well, many of our tree problems could have been caught way earlier | 11:26 |
| f13 | before they were ever composed into a tree | 11:26 |
| wwoods | (not to say it's not important, just want to make sure any specific goofs from F10 are examined and we have plans to prevent 'em recurring) | 11:26 |
| wwoods | could you give an example? | 11:27 |
| f13 | the conflicts could have been found | 11:27 |
| f13 | they've existed for a while, any check of the rawhide compose would have found them | 11:27 |
| -!- che [n=rkastl@redhat/che] has quit [Remote closed the connection] | 11:28 | |
| wwoods | OK, cool | 11:29 |
| wwoods | so, I assume we're talking about a qpid instance here? | 11:29 |
| wwoods | sending out messages when certain events occur? | 11:29 |
| f13 | yeah, that's my thoughts | 11:30 |
| f13 | the other side of that is somethign reacting to the messages | 11:30 |
| wwoods | so the first Proof of Concept will be having verifytree run when Rawhide finishes? | 11:30 |
| -!- XulChris [i=chris@fedora/XulChris] has joined #fedora-meeting | 11:30 | |
| -!- J5 [n=quintice@c-76-24-17-105.hsd1.ma.comcast.net] has quit [Read error: 60 (Operation timed out)] | 11:31 | |
| -!- openpercept [n=openperc@unaffiliated/openpercept] has left #fedora-meeting [] | 11:31 | |
| f13 | that seems like a useful thing to do | 11:32 |
| f13 | I was also going to target replacing the cvs post-commit email thing with just a message blast to the bus, and something else on the bus listening for that and taking care of generating email, rss, etc... | 11:33 |
| f13 | in theory it'll make cvs commits go much faster for the end user, fix the ^c hole (or at least make it much much smaller), and have more flexibilty in what we do with that information. | 11:33 |
| -!- mbacovsk [n=mbacovsk@nat/redhat/x-2a2a4f3ed4854b32] has quit [Read error: 60 (Operation timed out)] | 11:33 | |
| jds2001 | rss would be cool :) | 11:33 |
| wwoods | interesting stuff. can you keep us posted on how that's progressing? esp. if you've got a test instance up or some design docs we can write code against | 11:33 |
| -!- dwmw2_yoyo [i=ctrlprox@baythorne.infradead.org] has quit [Remote closed the connection] | 11:33 | |
| f13 | yep | 11:34 |
| f13 | Hopefully this week I'll have my homeserver set up enough to launch an el5 guest to start tinkering again | 11:35 |
| wwoods | awesome. | 11:35 |
| f13 | then I"ll move on to a publictest instance | 11:35 |
| -!- fab [n=bellet@monkey.creatis.insa-lyon.fr] has quit ["Leaving"] | 11:35 | |
| wwoods | okay. any other stuff specific to F10 we should talk about fixing up? | 11:35 |
| -!- mcepl [n=mcepl@nezmar.jabbim.cz] has left #fedora-meeting [] | 11:35 | |
| f13 | let me look at my list | 11:36 |
| f13 | oh. | 11:36 |
| f13 | Daily compose attempts of Live images, and information gathering of them | 11:36 |
| f13 | trying to make "oh shit it doesn't fit" emergencies happen less | 11:36 |
| wwoods | ahhh yes. | 11:37 |
| f13 | this wasn't so much an issue toward the end of the cycle, more of an issue around alpha/beta | 11:37 |
| -!- dwmw2 [i=ctrlprox@baythorne.infradead.org] has joined #fedora-meeting | 11:37 | |
| wwoods | plus it gives us more live images to test | 11:37 |
| f13 | I've got a pile of code written up, I need to finish it and deploy it | 11:37 |
| wwoods | err more up-to-date | 11:37 |
| f13 | I'm not sure if the images themselves will be posted | 11:37 |
| f13 | that's a /ton/ of churn | 11:37 |
| f13 | (these are not rsync friendly files) | 11:37 |
| f13 | but the output from the compose attempts and various info regarding the images will be kept around | 11:38 |
| wwoods | true. so just a build-and-check-for-explosions? | 11:38 |
| jds2001 | i thought we agreed not to post the images. | 11:38 |
| wwoods | perhaps an attempt to boot 'em in KVM or something | 11:38 |
| -!- mcepl [n=mcepl@nezmar.jabbim.cz] has joined #fedora-meeting | 11:38 | |
| -!- dwmw2 is now known as dwmw2_yoyo | 11:38 | |
| jds2001 | that might be nice, if easily automatable (i'm sure it is) | 11:38 |
| wwoods | anyway, yes, they don't need to get posted, I was just pondering | 11:38 |
| viking_ice_ | is it possible to release iso spins more often that seems to be adding in testing stuff perhaps weekly releases.. | 11:38 |
| f13 | first target is just attempt to compose, gather info, diff against previous day/milestone | 11:39 |
| viking_ice_ | users seems to be more willing to test if they have an iso to play with.. | 11:39 |
| f13 | beyond that we can talk about adding more stuff in, like kvm trials. | 11:39 |
| f13 | viking_ice_: we release the /tools/ that we use to make isos, so that/anybody/ can make isos at any time | 11:39 |
| viking_ice_ | I'm aware of that.. | 11:39 |
| -!- Evil_Sonar_Chick [n=Andrea@fedora/SonarGal] has joined #fedora-meeting | 11:39 | |
| f13 | viking_ice_: trying to host isos for download or mirror them more frequently will slow /everything/ down. | 11:40 |
| viking_ice_ | they arent or not willing to use them.. | 11:40 |
| viking_ice_ | f13 solved by torrent only | 11:40 |
| f13 | not really | 11:40 |
| wwoods | not really solved, no | 11:40 |
| f13 | we get a ton of complaints that torrent misses wide swaths of our userbase | 11:40 |
| wwoods | torrents only work when you have a *lot* of people requesting the same file | 11:40 |
| wwoods | and you're not firewalled off from using it | 11:41 |
| f13 | and the more frequent iso drops means less people on the torrent which means slower downloads for everybody | 11:41 |
| jds2001 | one other thing is that you have to *have* rawhide to compose a rawhide live image | 11:41 |
| f13 | jds2001: or a chroot thereof | 11:41 |
| wwoods | no, I think people just need to be instructed on how to use boot.iso | 11:41 |
| jds2001 | not an issue for me since I have a machine that runs rawhide all the time., | 11:41 |
| -!- Sonar_Gal [n=Andrea@fedora/SonarGal] has quit [Read error: 60 (Operation timed out)] | 11:41 | |
| f13 | (well, except for the livecd case, livecds can't be composed in chroots *grrr*) | 11:41 |
| jds2001 | f13: didnt think chroot worked for livecd-creator | 11:41 |
| -!- hpachas-PE [n=hpachas@200.37.120.18] has joined #fedora-meeting | 11:42 | |
| f13 | we could however guide people into making their own iso sets for test days | 11:43 |
| f13 | and have better examples on the wiki of how to make iso sets from rawhide for extended testing | 11:43 |
| viking_ice_ | well I basically want to expand the idea of test day and make every day a test day.. | 11:43 |
| jds2001 | yeah, i have a blog post that i've been meaning to generalize into a wiki page | 11:44 |
| -!- hanthana [n=hanthana@124.43.55.58] has joined #fedora-meeting | 11:44 | |
| * jds2001 adds that to the pile o' doom. | 11:44 | |
| wwoods | more test days is a good idea, but we'll need more testers if we want to have more of 'em | 11:44 |
| wwoods | so yes, lowering the barrier to entry for testing is a good idea | 11:44 |
| -!- Nirmal [n=npathak@nat/redhat-in/x-e7c02fb6a638ea48] has joined #fedora-meeting | 11:45 | |
| wwoods | but, as mentioned, I don't think we're going to be able to get full composes daily | 11:45 |
| wwoods | and instead we should focus on simplifying/documenting making yer own images | 11:45 |
| -!- bo09 [n=bo09@nat/redhat/x-8c977667aea562d7] has joined #fedora-meeting | 11:45 | |
| viking_ice_ | i'm not that worried about it just though I mention it + iso are only needed for install testing so... | 11:45 |
| f13 | iso are only needed for iso based install testing | 11:46 |
| wwoods | yeah, but the most part of install testing can be done with boot.iso | 11:46 |
| f13 | you can do lots of other install testing without full .isos | 11:46 |
| wwoods | or with network installs | 11:46 |
| -!- J5 [n=quintice@c-76-24-17-105.hsd1.ma.comcast.net] has joined #fedora-meeting | 11:46 | |
| * jds2001 generally uses PXE and network method o' the day. | 11:46 | |
| jds2001 | NFS/HTTP/FTP | 11:46 |
| viking_ice_ | it's faster as well | 11:47 |
| jds2001 | i also have a local mirror. | 11:47 |
| -!- neverho0d [n=psv@62.68.142.123] has quit [Read error: 60 (Operation timed out)] | 11:47 | |
| wwoods | so, yes, to summarize: more documentation / better tools for doing installs (esp. in KVM) would be really helpful in getting more testers | 11:48 |
| -!- balor [n=balor@gimili.plus.com] has quit ["Ex-Chat"] | 11:49 | |
| f13 | and to echo clumens, screw more testers, I want more fixers (: | 11:49 |
| wwoods | jds2001 is going to (eventually) write a wiki page on creating your own CD/DVD images | 11:50 |
| wwoods | I'll write a wiki page on using boot.iso/netinst.iso to do a network install, I guess? would that be helpful? | 11:50 |
| wwoods | or do we think that everyone who is *willing* to install rawhide already knows how? | 11:51 |
| viking_ice_ | Has not any of the programmers tried reeling in young programmers at uni's and have some kind of mentoring project ? | 11:51 |
| wwoods | we do have a couple mentoring projects actually | 11:51 |
| f13 | Red Hat has just started getting into some of that | 11:51 |
| viking_ice_ | wwoods: Think that would be setup under QA under Setting up you own test system | 11:52 |
| f13 | but usually it's a quagmire of paperwork | 11:52 |
| viking_ice_ | beer stopped working.. | 11:52 |
| sgtd | heh | 11:52 |
| f13 | and trying to find something that fits will with a students class load | 11:52 |
| wwoods | Chris Tyler - who's on the Fedora Board - is a professor at Seneca College in Toronto | 11:52 |
| viking_ice_ | wwoods: + i want to split testing into level beginner amateur expert debugger even.. | 11:53 |
| wwoods | viking_ice_: yeah, having difficulty ratings for different test/QA tasks would be helpful | 11:53 |
| wwoods | so let's talk about wiki stuff some more | 11:54 |
| jds2001 | but where does a pungi compose fit into that? That's trivial and I can do it in my sleep. But it might not be for someone else. | 11:54 |
| f13 | jds2001: it's pretty trivially documented or walked through | 11:54 |
| wwoods | jds2001: I'd rate that as fairly advanced, since it requires some infrastructure and a load of disk and time and *then* you have to burn and install.. | 11:54 |
| viking_ice_ | depends on the doc if just anyone else can do that :) | 11:54 |
| f13 | jds2001: the first time is the hardest, but the hardest part is gleaning the difference between a "I did the compose wrong" and "the tree is busted" errors. | 11:55 |
| wwoods | it's not as easy as, say, enabling updates-testing | 11:55 |
| viking_ice_ | which would be for beginners.. | 11:55 |
| -!- sonar_logger3 [n=Who@unaffiliated/sonarlogger1/x-872231] has quit [Read error: 110 (Connection timed out)] | 11:55 | |
| -!- abadger19991 [n=abadger1@65.78.187.8] has joined #fedora-meeting | 11:55 | |
| -!- abadger1999 [n=abadger1@65.78.187.8] has quit ["Leaving."] | 11:55 | |
| -!- sonar_logger3 [n=Who@unaffiliated/sonarlogger1/x-872231] has joined #fedora-meeting | 11:56 | |
| wwoods | it's a big question, but we definitely have some wiki grooming to do | 11:56 |
| wwoods | which leads me to.. | 11:56 |
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA Meeting | Wikified Test Plans | 11:57 | |
| wwoods | We've talked about this a few times before - we were going to use the wiki to store test plans, then testopia, then it went away | 11:57 |
| wwoods | tracking test *execution* remains a different problem | 11:57 |
| viking_ice_ | just noting that wiki is no place to report on test plans.. | 11:57 |
| wwoods | exactly | 11:57 |
| wwoods | filling in the matrixes on the wiki sucks. but we'll talk about that some other day. | 11:57 |
| jds2001 | dmalcolm was supposed to come up wiht something bigger/faster/better, but that's in the planning stages atm iirc | 11:58 |
| viking_ice_ | and follows kiss | 11:58 |
| jds2001 | though i did see his name on the fudcon list :) | 11:58 |
| wwoods | yeah, I think there will be some stuff to discuss in that regard at fudcon | 11:58 |
| jds2001 | which btw, are you going wwoods? I didn't see or overlooked your name. | 11:59 |
| wwoods | but as for writing test *plans* and related stuff.. | 11:59 |
| wwoods | oh, um | 11:59 |
| wwoods | I'm planning on it. Am I not on the list? | 11:59 |
| wwoods | nope, there I am, #30 | 11:59 |
| wwoods | anyway | 12:00 |
| viking_ice_ | well i'm seeing ( simpler ) version of bodhi as a reporting how it went tho the actually testplan could be on the wiki | 12:00 |
| jds2001 | ahh, i'm blind. Never mind me.... | 12:00 |
| -!- neverho0d [n=psv@62.68.142.121] has joined #fedora-meeting | 12:00 | |
| wwoods | right - regardless of what we have to track test reporting, we need to put plans somewhere | 12:01 |
| wwoods | and we've already got this wiki | 12:01 |
| wwoods | so. here's the proposal. | 12:01 |
| wwoods | 1) create a QA: namespace. | 12:01 |
| wwoods | 2) start creating a page per-package under QA: - like QA:kernel | 12:01 |
| wwoods | or QA:rhythmbox | 12:02 |
| -!- nphilipp [n=nils@155.56.40.12] has quit ["Leaving"] | 12:02 | |
| wwoods | 3) each package-page will contain several sections, such as: | 12:02 |
| wwoods | a) How to test (which could contain links to pages for other test plans) | 12:02 |
| wwoods | b) Troubleshooting | 12:03 |
| wwoods | c) Reporting a bug | 12:03 |
| wwoods | now. someday in the glorious future we should also have a bug-reporting client program | 12:03 |
| viking_ice_ | I think you can skip b since it should be in a | 12:03 |
| jds2001 | wwoods: we already do, python-bugzilla :) | 12:04 |
| viking_ice_ | I would rather see it as install test develope | 12:04 |
| * jds2001 ducks | 12:04 | |
| wwoods | which will display the b) section when you want to report a bug about that package | 12:04 |
| wwoods | (or something like that) | 12:04 |
| -!- than [n=than@nat/redhat/x-8b24c19c42058d84] has quit [Remote closed the connection] | 12:04 | |
| wwoods | it'd also be good to have a subsection of c) that lists files (and their mime-types) in a machine-readable format | 12:04 |
| wwoods | so the bug-reporter client can automatically attach those files to a bug report | 12:04 |
| viking_ice_ | also we need to create a template for that if we are going to fetch info from it into bugzilla.. | 12:04 |
| wwoods | viking_ice_: what do you mean by "install test develope"? | 12:05 |
| -!- XulChris [i=chris@fedora/XulChris] has left #fedora-meeting [] | 12:05 | |
| wwoods | yeah, definitely going to need templates and examples | 12:05 |
| wwoods | But there's a difference between having a section on "how to test a new rhythmbox package to make sure it works" and "things to try if rhythmbox is acting funny" | 12:07 |
| -!- XulChris [i=chris@fedora/XulChris] has joined #fedora-meeting | 12:07 | |
| wwoods | e.g. the first thing a triager will say on any xorg bug report is: "can you try running without xorg.conf" | 12:07 |
| viking_ice_ | How to install <--> A from package b from source ( svn ) How to test <--> reporting step by step guidance test cases what to test etc How to develop <--> developing info | 12:07 |
| jds2001 | why would we install anything from source? | 12:08 |
| * jds2001 confuzzled, we're testing actual builds here. | 12:08 | |
| viking_ice_ | upstream testing done some of those with projects on fedorahosted.. | 12:08 |
| jds2001 | oh, sure - upstream can do what they like :) | 12:09 |
| jds2001 | even if we're upstream, they've gotta package it to get it in Fedora. | 12:09 |
| viking_ice_ | you get faster testing reporting with svn checkouts.. you test he codes you test again.. more efficient | 12:09 |
| viking_ice_ | better results.. | 12:10 |
| jds2001 | right, that goes with working upstream. | 12:10 |
| jds2001 | we're talking about our distro atm, not working with every single upstream to QA their code. | 12:10 |
| viking_ice_ | hence I personally stuck with fedorahosted.. | 12:11 |
| * jds2001 not planning on building GNOME anytime soon, for example. | 12:11 | |
| viking_ice_ | should be appealing argument if we are upstream that we provide testers.. | 12:11 |
| viking_ice_ | ofcourse fedora reaps the benefit of that effort.. | 12:12 |
| jds2001 | we can't even test the integration of our own friggin distro, let's not branch out into upstream quite yet :) | 12:12 |
| jds2001 | someone stop me if you disagree :) | 12:12 |
| wwoods | You make a good point, though - it makes fedorahosted more attractive for someone looking for hosting | 12:13 |
| wwoods | if we say "look, you get a whole bunch of testers for free" | 12:13 |
| viking_ice_ | well there are issues we need to overcome that get always thrown back and fourth I say build a community another say more testers means more reports and we cant handle the reports as is .. | 12:13 |
| viking_ice_ | etc.. | 12:14 |
| wwoods | what we really need is *smarter* testers and *better* bug reports | 12:14 |
| viking_ice_ | but first step is cleaning up wiki so lets get that QA name space.. | 12:14 |
| wwoods | which means - give people more docs and tools that help them report better bugs | 12:14 |
| f13 | (and more people fixing things) | 12:14 |
| viking_ice_ | and testers dont get smarter if they dont have docs to read.. | 12:14 |
| f13 | always the missing part of the equasion. | 12:14 |
| -!- abadger19991 [n=abadger1@65.78.187.8] has quit [Read error: 148 (No route to host)] | 12:15 | |
| wwoods | f13: there's two responses to that: 1) Hey, we're QA! that's not our problem! | 12:15 |
| wwoods | but much better is: 2) Finding the problem is half of fixing it | 12:15 |
| wwoods | so giving our testers better tools / guidance on how to find the *actual* cause of their problems | 12:16 |
| wwoods | will make things easier on current devs | 12:16 |
| wwoods | and probably turn some of those testers into fixers | 12:16 |
| -!- jtibu [n=jtibu@uniaraxa.edu.br] has joined #fedora-meeting | 12:16 | |
| f13 | one could only hope (: | 12:17 |
| wwoods | It certainly can't hurt | 12:17 |
| wwoods | It took me a long time of fumbling around and getting hints from people on how to trace problems / write better bug reports | 12:18 |
| wwoods | before I got to the point of being able to track down problems fully myself and actually generate patches to fix 'em | 12:18 |
| wwoods | I think leading people along that path will have its benefits | 12:18 |
| wwoods | anyway, yeah | 12:18 |
| wwoods | ACTION: wwoods will request that infrastructure set up the QA namespace on the wiki | 12:18 |
| viking_ice_ | starting with beginner --> turning into advanced --> expert --> debugger --> developing.. | 12:19 |
| wwoods | exactly | 12:19 |
| viking_ice_ | all in the master plan.. | 12:19 |
| wwoods | and then the last step: total world domination | 12:19 |
| viking_ice_ | takes time deploying this but we can expect to see actually results 2 release from now.. | 12:19 |
| wwoods | anyway once we have the namespace, viking_ice_, do you have a certain package you want to use as an example/template? | 12:19 |
| wwoods | it'd be good to have a couple proposed example QA pages that we can use to write up a full specification | 12:20 |
| wwoods | first it might be good to list the actual requirements, which I think are: a) how to test, b) how to report bugs | 12:21 |
| viking_ice_ | I can create one i've been meaning to start with X | 12:21 |
| wwoods | c) developer notes (building etc.) might be optional | 12:21 |
| wwoods | section b) probably should have a subsection that describes the files needed in a proper bug report | 12:21 |
| viking_ice_ | That's something that a maintainer needs to provide.. | 12:21 |
| wwoods | we also need to think about what do to for packages that share test plans etc. (probably redirect them to the Master Page) | 12:22 |
| viking_ice_ | for instance lot of triagers and other testers have been asking reporters to attach file that they did not need nor want | 12:22 |
| -!- giallu [n=giallu@fedora/giallu] has quit [Remote closed the connection] | 12:22 | |
| wwoods | right, we want the developers to know that they can (and should) edit the QA pages themselves | 12:23 |
| wwoods | but first we need to figure out what is required and what is optional | 12:23 |
| viking_ice_ | atleast they need to approve them.. | 12:23 |
| jds2001 | no way we can write test plans for ~4000 components. | 12:23 |
| jds2001 | but a maintainer can write a plan for say, 10. | 12:24 |
| viking_ice_ | jds2001: falls under the "every day is a test day ... " | 12:24 |
| wwoods | yeah, and I don't really expect there to be test plans for everything | 12:24 |
| wwoods | but guidance on the really common/important stuff would be a great boon | 12:24 |
| * jds2001 would like to start with a package I own in EPEL that never gets touched, the most important package of all though! | 12:25 | |
| jds2001 | cowsay! :) | 12:25 |
| jds2001 | not really, just figured I'd get my cowsay promo out there. | 12:25 |
| wwoods | ha | 12:25 |
| viking_ice_ | maintainer: sends us with a day or 2 in advanced on what he wants to be tested we prepare the wiki for our testers mark the test-ing on the calender and post to the test list what to test.. | 12:26 |
| wwoods | that's a good way to build up the test plans, definitely | 12:26 |
| -!- abadger1999 [n=abadger1@65.78.187.8] has joined #fedora-meeting | 12:27 | |
| viking_ice_ | thats the finally thingi with every day is a test day + it ensures that all the components which aren't very popular or winning any beauty patents get they're fair share of testing | 12:28 |
| -!- paulobanon_ [n=psantos@ns.alphatim.org] has quit [Read error: 145 (Connection timed out)] | 12:30 | |
| wwoods | do we want to plan a rawhide test day now? or maybe an F10 test day to clear up some outstanding issues? | 12:30 |
| viking_ice_ | the problem is that it does not fix the problem with the final out we need to gets some intel from forums and #fedora and focus on that.. then get the unity guys to issue an respin.. | 12:33 |
| viking_ice_ | then start focusing on rawhide again.. | 12:33 |
| -!- fbijlsma [n=fbijlsma@p54B2E1D7.dip.t-dialin.net] has joined #fedora-meeting | 12:34 | |
| wwoods | so we'll focus on F10 issues for a little while.. sounds like a good plan\ | 12:34 |
| viking_ice_ | F10 Test day would be better focusing on what's being reported.. | 12:34 |
| viking_ice_ | Are we doing any data mining on #fedora and the fedora forum and the fedora-user list ? | 12:37 |
| wwoods | I'm not, personally | 12:37 |
| f13 | I think we should give rawhide a little bit of time to smooth out from the libtool/python/rpm pkg-config fallout | 12:37 |
| wwoods | I've only got so much time in the day | 12:37 |
| f13 | although a test-day that covers those things would be useful. | 12:37 |
| -!- lfoppiano [n=lfoppian@fedora/lfoppiano] has quit ["Ex-Chat"] | 12:38 | |
| viking_ice_ | f13: true get testing when python 2.6 is done.. | 12:38 |
| jds2001 | f13: can ivazquez tell us when he thinks python is in a stable state? | 12:38 |
| jds2001 | i.e. lots of things using it? | 12:38 |
| f13 | I'm sure he can. | 12:39 |
| viking_ice_ | arent there any responsible for the #fedora and fedora forum along with fedora-user list? could we please get the test-traffic of the devel list to the test list | 12:39 |
| ivazquez | We'll notice it get stable when the Rawhide reports stop showing so many "python(abi)" and "libpython2.5" dep problems. | 12:39 |
| f13 | viking_ice_: sounds like you're volunteering | 12:40 |
| wwoods | There's a few people that do a #fedora op helper thing, occasionally they let me know about stuff that people are complaining about a lot | 12:41 |
| wwoods | but nothing official | 12:41 |
| viking_ice_ | volunteering on what ? ( me doing wiki stuff ) Rahul was good at this along with Andrew I think during F7 or F8 bouncing the reporters to the test list.. | 12:41 |
| wwoods | generally they get told "go file a bug" | 12:41 |
| viking_ice_ | We need to activate them in statistic... | 12:42 |
| -!- stickster is now known as stickster_food | 12:42 | |
| viking_ice_ | priorities on that with a test day,, | 12:42 |
| mcepl | BTW, there is planned (but not yet announced) perl fallout | 12:42 |
| mcepl | probably not as brutal, but affecting many (if not most) perl-* packages | 12:43 |
| -!- j-rod [n=j-rod@static-72-93-233-3.bstnma.fios.verizon.net] has joined #fedora-meeting | 12:43 | |
| viking_ice_ | will X 1.6 be introduced in F11 + we need to get extensive testing on ext4 | 12:43 |
| viking_ice_ | something tells me it's going to be default now.. | 12:43 |
| jds2001 | viking_ice_: i dont see a feature for that yet, it surely will be one. | 12:44 |
| jds2001 | same with any Xorg rebase | 12:45 |
| viking_ice_ | and another thing will kernel 29 be inn final.. | 12:45 |
| wwoods | dunno about X 1.6 - that's an ajax question | 12:45 |
| wwoods | same with kernel. probably yes to both. | 12:45 |
| viking_ice_ | I pop the question did not get a reply.. | 12:45 |
| jds2001 | viking_ice_: they're looking at 2.6.30 I thought. | 12:45 |
| wwoods | maybe kernel .30. but who knows? it's hard to say. | 12:45 |
| f13 | "If it's ready in time" | 12:46 |
| viking_ice_ | well anything other than whats current.. | 12:46 |
| wwoods | I don't think we'll switch to ext4 by default | 12:46 |
| wwoods | although it would be fun to drop LVM *and* do ext4 by default | 12:46 |
| viking_ice_ | why not? | 12:46 |
| wwoods | just to really mess with people | 12:46 |
| viking_ice_ | if its going to make into RHEL 6 it needs to be tested.. | 12:46 |
| viking_ice_ | now.. | 12:47 |
| wwoods | because it's unproven and has no significant benefits over ext3 for nearly all Fedora users | 12:47 |
| jwb | btrfs | 12:47 |
| f13 | it's like buttah! | 12:47 |
| wwoods | OTOH it's quite stable and somewhat quicker, and it's not a huge change | 12:47 |
| jds2001 | FESCo is in 10 minutes, and I'd like to get something to eat real quick :) | 12:47 |
| wwoods | I can see both sides of the argument. I'll leave it up to sandeen and other very smart folks | 12:47 |
| f13 | I'd be in favor of ext4 by default, if we made the switch pre-alpha | 12:47 |
| wwoods | but they should definitely decide soon.. yeah exactly | 12:48 |
| wwoods | it'd be a Feature | 12:48 |
| wwoods | should we poll them on that? | 12:48 |
| jwb | yes | 12:48 |
| wwoods | "QA requests that this be decided immediately to allow for maximum test time" | 12:48 |
| -!- petreu| [n=peter@p3EE3DDCF.dip.t-dialin.net] has joined #fedora-meeting | 12:48 | |
| viking_ice_ | yup sounds good.. | 12:48 |
| wwoods | also, yes, lunch | 12:48 |
| viking_ice_ | lunch.. ? | 12:48 |
| jwb | FOOD IS OVERRATED | 12:48 |
| viking_ice_ | sleep to.. | 12:48 |
| wwoods | lunch/sleep, depending on your time zone | 12:49 |
| wwoods | heh | 12:49 |
| viking_ice_ | wwoods: QA namespace.. | 12:49 |
| wwoods | right, QA namespace | 12:49 |
| viking_ice_ | that's the first stone.. | 12:49 |
| wwoods | and then you and I will cook up some example pages | 12:49 |
| viking_ice_ | definitely | 12:49 |
| wwoods | and we'll send mail to fedora-test-list and/or fedora-devel-list for discussion | 12:49 |
| wwoods | to see if there's any requirements we're missing | 12:50 |
| wwoods | mostly we'll just get a lot of useless bikeshed painting but that's OK | 12:50 |
| viking_ice_ | yup after we make the examples... | 12:50 |
| * f13 goes for breakfast | 12:50 | |
| viking_ice_ | i'm close to supper... | 12:50 |
| wwoods | ONGOING: continue rounding up F10 bugs, consider a test day for clearing out bug backlog | 12:51 |
| f13 | and wwoods is off for lunch. THe trifecta! | 12:51 |
| f13 | wwoods: oh, I subscribed to the rawhide bugs rss feed, so I'm having a much better eye on what's happening there. | 12:51 |
| wwoods | ACTION: ask ivasquez to notify QA when the Great Python Updating is complete so we can try a test day | 12:51 |
| -!- mether [n=sundaram@122.169.93.162] has joined #fedora-meeting | 12:51 | |
| f13 | after the initial slug through the backlog, it's going to be much easer to keep up on the incoming stuff. | 12:51 |
| * viking_ice_ wants python 3.0 | 12:51 | |
| wwoods | ACTION: ask sandeen / kernel devs / etc. to make decision about ext4-by-default in F11 ASAP so we can get testing docs ready | 12:52 |
| * jwb goes to talk to sandeen | 12:52 | |
| wwoods | anything else before food/sleep overtake us all? | 12:52 |
| * wwoods meeting end in 30s | 12:53 | |
| * wwoods meeting end in 15s | 12:53 | |
| wwoods | Meeting over. Whew. Thanks very much for your time, everyone | 12:54 |
| --- Log closed Wed Dec 03 12:54:12 2008 | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!