| --- Log opened Wed Jan 14 11:03:08 2009 | ||
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA | init | 11:03 | |
| wwoods | jds2001, f13, jlaska, poelcat: ping | 11:04 |
|---|---|---|
| * jds2001 here | 11:04 | |
| * jlaska here (in another meeting) | 11:05 | |
| wwoods | really wanted f13 here, since he's got the notes from the QA hackfest | 11:06 |
| -!- Zool^ [n=kaland@213.225.96.164] has quit [Remote closed the connection] | 11:07 | |
| wwoods | anyone else here? like, anyone who *wasn't* at FUDCon? | 11:07 |
| jlaska | wwoods: http://fpaste.org/paste/1378 | 11:08 |
| wwoods | well, either way, let's recap / summarize the FUDCon discussions | 11:08 |
| wwoods | jlaska: ah, excellent, thanks | 11:08 |
| -!- lvillani [n=lvillani@kde/developer/lvillani] has left #fedora-meeting ["Leaving"] | 11:09 | |
| wwoods | so let's talk about autoqa first | 11:09 |
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA | FUDCon Recap | AutoQA | 11:09 | |
| -!- openpercept [n=openperc@unaffiliated/openpercept] has joined #fedora-meeting | 11:09 | |
| wwoods | it's long past time to have some automated checks at various points in the build process. | 11:10 |
| wwoods | so, we identified a list of triggers - places in the process where we want to be able to run tests | 11:10 |
| wwoods | http://fpaste.org/paste/1378 has them, under "What can we verify now?/event driven" | 11:11 |
| wwoods | we're going to have a defined environment for each trigger | 11:12 |
| wwoods | and then you can write tests - for example: | 11:12 |
| wwoods | "bodhi request" - after a bodhi request is made, we'd like to run some checks on the new package | 11:12 |
| wwoods | we'll probably need to pass the script, say, the URL of the koji server, the new build NVR.. something along those lines | 11:13 |
| wwoods | given that info you can perform useful checks like.. comparing the new package(s) to the previous version | 11:13 |
| wwoods | making sure the new packages can be installed successfully | 11:14 |
| wwoods | etc. | 11:14 |
| wwoods | we'll kind of make it up as we go - and I'll need to discuss this with bpeck and other automated test experts inside RH | 11:15 |
| wwoods | because ideally we want all the tests using the same conventions for reporting success/failure, etc. | 11:15 |
| wwoods | anyway, jds2001 did the legwork to get a git repo created: http://git.fedorahosted.org/git/?p=autoqa.git;a=summary | 11:15 |
| wwoods | if you want/need access to it, let me know | 11:16 |
| wwoods | I'm going to create directories for each trigger and start putting tests in there as we write them | 11:16 |
| wwoods | along with documentation of the test environment for each trigger | 11:16 |
| wwoods | actually getting the tests hooked into the infrastructure will be something that we'll work out with rel-eng and infra | 11:17 |
| wwoods | initially the results will probably just get emailed to various places | 11:17 |
| wwoods | but, again, I'd like to see a unified library that provides each test with different ways to report status - email, sending dbus / amqp messages, etc | 11:18 |
| wwoods | Now, one of the triggers.. which might not be on that list at the moment | 11:19 |
| wwoods | but anyway - post-install tests. because, yes, we plan to finally do automated installs of rawhide daily | 11:19 |
| wwoods | and have a rawhide status page | 11:20 |
| wwoods | this will be built on the lab-in-a-box stuff that jlaska's been working on | 11:20 |
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA | FUDCon Recap | lab-in-a-box | 11:20 | |
| wwoods | jlaska: do you have time to give a quick summary of that stuff? or a link? | 11:20 |
| jds2001 | so there's stuff at jlaska.livejournal.com on how to set it up. | 11:22 |
| jlaska | sure ... sorry distracted | 11:22 |
| wwoods | well, that's OK, I can try to fake it | 11:22 |
| wwoods | heh | 11:22 |
| jds2001 | take it away jlaska :) | 11:22 |
| jlaska | jds2001: thanks, yeah, I'm posting my experiences in jlaska.livejournal.com | 11:22 |
| jlaska | I'll craft up a post tomorrow on how to use SNAKE kickstart templates within the the cobbler env | 11:22 |
| -!- mxcarron [n=maxime@fedora/Pingoomax] has quit [Read error: 104 (Connection reset by peer)] | 11:23 | |
| -!- mxcarron [n=maxime@fedora/Pingoomax] has joined #fedora-meeting | 11:24 | |
| wwoods | the basic lab-in-a-box concept is putting together a set of software (cobbler, koan, snake) to allow really, really easy test-installation of Fedora releases | 11:24 |
| jlaska | I've been discussing with some folks in the last few days to determine the best mechanism to "package" this | 11:24 |
| wwoods | in a "lab" as small as two systems with a crossover cable, or one system with virtualization capabilities | 11:24 |
| jlaska | for my purposes, this is all on one system, but uses the same utilities as beaker and can support additional hardware given certain criteria are met | 11:25 |
| wwoods | remote power control? serial console muxes? | 11:27 |
| f13 | sorry, here now. | 11:27 |
| f13 | got a little late start this morning. | 11:27 |
| jlaska | right right, and I can start to craft up more blog/howto's for stuff like that | 11:27 |
| f13 | jlaska: I haven't read through, but do you have details on being able to throw arbitrary trees at cobbler, like say test composes each day? | 11:27 |
| jlaska | f13: that's something that we quickly hacked up at the lunch'n'learn | 11:28 |
| f13 | and how to purge off cobbler's view of those trees so that when you purge the tree, cobbler doesn't get upset? | 11:28 |
| jlaska | I'll get details on that posted soon | 11:28 |
| MooDoo | .wc | 11:28 |
| -!- MooDoo [n=paulmell@cpc2-mapp2-0-0-cust80.nott.cable.ntl.com] has left #fedora-meeting [] | 11:28 | |
| jlaska | bpeck has some helpers for that ... I need to talk w/ mpdehaan hopefully on Friday to determine some packaging issues (whether to create a new tool, or push stuff into other existing projects) | 11:29 |
| f13 | k | 11:30 |
| wwoods | so this will let us do automatic rawhide installs - very simple installs at first, progressing to more complex tests as time/hardware capablity allows | 11:30 |
| wwoods | and should also allow us to do more automated install testing on releases | 11:30 |
| wwoods | anything else we need to cover on lab-in-a-box? | 11:31 |
| jlaska | that's all I've got right now, I'll likely have more updates next week | 11:31 |
| wwoods | cool. look forward to hearing about it! | 11:32 |
| wwoods | the next thing on my list is automated / simplified problem reporting | 11:33 |
| wwoods | which is kind of a mouthful | 11:33 |
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA | FUDCon Recap | oops.fedoraproject.org | 11:33 | |
| wwoods | let's call it oops.fedoraproject.org for now | 11:33 |
| wwoods | the concept is this: we want to set up a system similar to kerneloops | 11:34 |
| wwoods | in that it allows automated and anonymous filing of problem reports | 11:34 |
| wwoods | but for everything in Fedora | 11:35 |
| wwoods | we'll gather aggregate data from this Big Oops Database and cherry-pick critical, common, or easy-to-fix problems and push them to Bugzilla | 11:35 |
| jds2001 | my guess is that most of it needs to go upstream. | 11:36 |
| wwoods | we'll have a simple "Report a problem" app for people to run on their systems | 11:36 |
| wwoods | yeah - there are chunks of this that are philisophically very similar to Apport | 11:36 |
| wwoods | and I'm thinking about giving in and extending the Apport reporting format to handle the things we want to do | 11:37 |
| wwoods | and/or using chunks of their UI | 11:37 |
| -!- mxcarron [n=maxime@fedora/Pingoomax] has quit [Read error: 104 (Connection reset by peer)] | 11:37 | |
| wwoods | they're trying to get that to be a distro-agnostic tool and it seems silly to reinvent the wheel | 11:37 |
| wwoods | but at the same time.. last I checked there were some questionable design decisions | 11:38 |
| jds2001 | yeah, i can't claim to have looked much at apport. | 11:38 |
| wwoods | the definition of "upstream" is unclear here, too | 11:39 |
| wwoods | since this is a tool for users to report bugs to the *distributor* of their software | 11:39 |
| wwoods | not the *authors* | 11:39 |
| wwoods | for example, enterprise customers will likely want to run their own oops.helpdesk.globocorp.biz server | 11:40 |
| -!- alexxed [n=alex@dyn-85.186.119.28.tm.upcnet.ro] has joined #fedora-meeting | 11:40 | |
| wwoods | so their IT department can filter stuff and report it to Fedora - or Red Hat - as appropriate | 11:40 |
| wwoods | so it doesn't make sense to try to push this into GNOME, or bug-buddy | 11:41 |
| wwoods | but it benefits all Linux distros to have a tool like this | 11:41 |
| jds2001 | right, i just mean that a lot of the data is going to have to go upstream (GNOME) at some point, but the Fedora/RHEL maintainers are in the best posistion to do that. | 11:41 |
| wwoods | oh! you mean most of the *data* needs to go upstream | 11:42 |
| wwoods | yes, that's absolutely correct - in a lot of cases we'll want to push the data from oops.fp.o up to GNOME | 11:42 |
| * wwoods brb | 11:45 | |
| -!- QuickStart [n=QUICKSTA@pool-71-172-136-28.nwrknj.east.verizon.net] has joined #fedora-meeting | 11:47 | |
| wwoods | sorry about that | 11:47 |
| wwoods | jds2001: yeah, I misunderstood - I thought you were saying this code needed to go upstream | 11:47 |
| wwoods | but yeah, we'll definitely want to be able to push collected data upstream to GNOME / kernel / freedesktop / etc. | 11:47 |
| wwoods | so this does involve writing a server-side to collect data | 11:48 |
| wwoods | a webUI to view the data | 11:48 |
| -!- openpercept [n=openperc@unaffiliated/openpercept] has quit [Read error: 104 (Connection reset by peer)] | 11:49 | |
| wwoods | client tools to catch crashes or just allow manual problem submission | 11:49 |
| jds2001 | ideally both automatic and manual. | 11:49 |
| wwoods | plus other stuff to, say, collect debuginfo to get proper tracebacks before submission | 11:49 |
| wwoods | err, yes, we want both automated and manual reporting | 11:50 |
| -!- J5 [n=quintice@c-66-31-41-146.hsd1.ma.comcast.net] has joined #fedora-meeting | 11:50 | |
| jds2001 | so ubuntu does that retracing serverside. | 11:50 |
| wwoods | right, which is a horrible security problem, since IIRC they upload the core file for later retracing | 11:50 |
| jds2001 | right, noticed that - have no idea how they get away with that. | 11:51 |
| f13 | users likely just don't know it's happening | 11:51 |
| f13 | (and ubuntu just doesn't care) | 11:51 |
| jds2001 | "hi, let me give you my bank account number!" | 11:51 |
| wwoods | it's opt-in, and most people aren't aware of what info is/could be in there | 11:51 |
| wwoods | oh plus they're not anonymous, AFAIK | 11:51 |
| wwoods | you need a launchpad cookie for it to work | 11:52 |
| wwoods | at least that was the case the last time I looked | 11:52 |
| wwoods | so that data is directly connected to your email address / launchpad identity | 11:52 |
| wwoods | we won't be doing that. | 11:52 |
| wwoods | so, a further step is to have default menu items / widgets to allow this from inside any app that wants to support it | 11:53 |
| wwoods | so you get unified problem reporting for, say, anaconda, setroubleshoot, GNOME, KDE, etc. | 11:54 |
| f13 | or at least a library for one | 11:54 |
| wwoods | GNOME and KDE are obviously harder problems but putting it into the apps that are RH upstream should be a good idea | 11:54 |
| wwoods | right | 11:54 |
| wwoods | library / glade files / etc. | 11:54 |
| wwoods | had some long talks with UI designer folks about what a tool like this should look like | 11:55 |
| wwoods | so I'd like to have one UI that didn't suck that everyone could use | 11:55 |
| wwoods | that would be nice. | 11:55 |
| wwoods | anyway. that's a much longer-term project than the autoqa stuff | 11:55 |
| wwoods | but it's something I talked about a lot. | 11:55 |
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA | FUDCon Recap | Bodhi changes | 11:56 | |
| wwoods | another thing we discussed a couple of times: bodhi changes | 11:56 |
| wwoods | The sound-bite version is: The updates system should be user-driven, not developer-driven | 11:56 |
| -!- mxcarron [n=maxime@fedora/Pingoomax] has joined #fedora-meeting | 11:57 | |
| wwoods | And we should pay closer attention to some packages than others | 11:57 |
| wwoods | basically we want to make some changes like.. instead of nagging developers when a package they built sits in bodhi | 11:57 |
| -!- J5 [n=quintice@c-66-31-41-146.hsd1.ma.comcast.net] has quit ["Ex-Chat"] | 11:58 | |
| wwoods | we *should* be sending mail to the reporters/cc lists of all the bugs listed | 11:58 |
| -!- J5 [n=quintice@c-66-31-41-146.hsd1.ma.comcast.net] has joined #fedora-meeting | 11:58 | |
| wwoods | if nobody cares enough to test the fix for the problem, there's no point in destabilizing things by nagging the maintainer to push out an untested package | 11:58 |
| wwoods | now, requiring more user/tester help to get updates out means that we may need to start tracking karma etc. for testers | 11:59 |
| -!- Matias_Arg [n=root@host14.190-31-208.telecom.net.ar] has joined #fedora-meeting | 12:00 | |
| jds2001 | we already do that , because the fact that it got pushed to testing is a comment in the bug. | 12:00 |
| -!- Matias_Arg [n=root@host14.190-31-208.telecom.net.ar] has left #fedora-meeting [] | 12:00 | |
| -!- giallu [n=giallu@fedora/giallu] has quit [Read error: 110 (Connection timed out)] | 12:00 | |
| wwoods | jds2001: true, but the wording may need to be changed to indicate the intent | 12:00 |
| jds2001 | right | 12:00 |
| wwoods | "this package will not be released until it is confirmed to fix this bug. Please test this package and confirm the fix here:" | 12:00 |
| wwoods | etc. | 12:00 |
| wwoods | now, obviously, this requires easier access to updates-testing | 12:01 |
| wwoods | ideally we want a simple PK-based tool that lets you plug in something like a Fedora update ID | 12:01 |
| wwoods | and will automatically: enable updates-testing, install that package and its dependencies, and remind you about it in a day or two | 12:02 |
| wwoods | (and/or let you revert to the pre-test version, if that update is broken) | 12:03 |
| jds2001 | reverting could be a pain, but I guess we'll have to figure out how to do it. | 12:04 |
| wwoods | that's something we're going to try to wrok out with hughsie. A lot of the backend bits are already in bodhi-client | 12:04 |
| wwoods | I think it's possible, but I'll need to discuss with skvidal and folks | 12:05 |
| wwoods | anyway, this is another longterm project | 12:06 |
| wwoods | autoqa takes priority | 12:06 |
| f13 | and above that, alpha takes priority | 12:07 |
| wwoods | ..is there anything else from FUDCon that I've missed? | 12:07 |
| wwoods | f13: indeed | 12:07 |
| wwoods | and I still have to fix preupgrade. boo. | 12:07 |
| wwoods | really need to find a co-maintainer (or hand that off to the anaconda team) | 12:07 |
| wwoods | So I think that's everything from FUDCon. There's a video of the QA talk: http://alt.fedoraproject.org/pub/alt/videos/2009/FUDConF11/qa.ogg | 12:08 |
| f13 | rawhide is in a bit of a ...state... right now. I'm working as hard as I can to identify the problems, and poke anaconda guys (pjones is who to bug right now for rawhide issues) to help debug and produce fixes | 12:08 |
| wwoods | and now, Alpha. | 12:09 |
| f13 | I /really/ hope to have a installable with kickstart rawhide by tomorrow | 12:09 |
| -!- wwoods changed the topic of #fedora-meeting to: Fedora QA | Rawhide (f11alpha) status | 12:09 | |
| wwoods | f13: anything you need help (re)testing? | 12:10 |
| -!- alexxed [n=alex@dyn-85.186.119.28.tm.upcnet.ro] has quit [Read error: 104 (Connection reset by peer)] | 12:10 | |
| -!- openpercept [n=openperc@unaffiliated/openpercept] has joined #fedora-meeting | 12:10 | |
| -!- alexxed [n=alex@dyn-85.186.119.28.tm.upcnet.ro] has joined #fedora-meeting | 12:11 | |
| f13 | wwoods: not yet. Current rawhide will bring up network in loader, if you retry after the first timeout. | 12:11 |
| f13 | I think that's a dbus issue there, but need to get with walters and dcbw to debug that part | 12:11 |
| f13 | next you'll need an updates.img from clumens if you're using kickstart as it has a parsing bug: http://clumens.fedorapeople.org/updates.img | 12:11 |
| f13 | after that I'm running into not being able to see my virt disk (virtio being used here) so I may have to move on to real hardware. | 12:12 |
| f13 | we need to get a wiki page up with the testing targets, so that we can do some pre-testing to see where we're broken. | 12:14 |
| -!- mxcarron [n=maxime@fedora/Pingoomax] has quit [Read error: 104 (Connection reset by peer)] | 12:14 | |
| wwoods | hm, okay. I guess I'll just carry over the old F10 final test list | 12:16 |
| wwoods | maybe drop all that wacky crypt stuff | 12:16 |
| wwoods | and add one or two simple crypt cases | 12:16 |
| f13 | yeah, for alpha I want to be able to get a basic install done | 12:17 |
| -!- spoleeba [n=one@fedora/Jef] has joined #fedora-meeting | 12:19 | |
| wwoods | after that, we'll see what we can do | 12:19 |
| wwoods | okay. anything else we need to discuss? | 12:20 |
| -!- linuxguru [n=linuxgur@unaffiliated/linuxguru] has quit [Read error: 110 (Connection timed out)] | 12:21 | |
| wwoods | alright then. meeting over, thanks for your time, folks | 12:21 |
| wwoods | log will be up shortly | 12:21 |
| --- Log closed Wed Jan 14 12:21:51 2009 | ||
Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!