--- Log opened Wed Jan 14 11:03:08 2009
-!- wwoods changed the topic of #fedora-meeting to: Fedora QA | init11:03
wwoodsjds2001, f13, jlaska, poelcat: ping11:04
* jds2001 here 11:04
* jlaska here (in another meeting)11:05
wwoodsreally wanted f13 here, since he's got the notes from the QA hackfest11:06
-!- Zool^ [n=kaland@] has quit [Remote closed the connection]11:07
wwoodsanyone else here? like, anyone who *wasn't* at FUDCon?11:07
jlaskawwoods: http://fpaste.org/paste/137811:08
wwoodswell, either way, let's recap / summarize the FUDCon discussions11:08
wwoodsjlaska: ah, excellent, thanks11:08
-!- lvillani [n=lvillani@kde/developer/lvillani] has left #fedora-meeting ["Leaving"]11:09
wwoodsso let's talk about autoqa first11:09
-!- wwoods changed the topic of #fedora-meeting to: Fedora QA | FUDCon Recap | AutoQA11:09
-!- openpercept [n=openperc@unaffiliated/openpercept] has joined #fedora-meeting11:09
wwoodsit's long past time to have some automated checks at various points in the build process.11:10
wwoodsso, we identified a list of triggers - places in the process where we want to be able to run tests11:10
wwoodshttp://fpaste.org/paste/1378 has them, under "What can we verify now?/event driven"11:11
wwoodswe're going to have a defined environment for each trigger11:12
wwoodsand then you can write tests - for example:11:12
wwoods"bodhi request" - after a bodhi request is made, we'd like to run some checks on the new package11:12
wwoodswe'll probably need to pass the script, say, the URL of the koji server, the new build NVR.. something along those lines11:13
wwoodsgiven that info you can perform useful checks like.. comparing the new package(s) to the previous version11:13
wwoodsmaking sure the new packages can be installed successfully11:14
wwoodswe'll kind of make it up as we go - and I'll need to discuss this with bpeck and other automated test experts inside RH11:15
wwoodsbecause ideally we want all the tests using the same conventions for reporting success/failure, etc.11:15
wwoodsanyway, jds2001 did the legwork to get a git repo created: http://git.fedorahosted.org/git/?p=autoqa.git;a=summary11:15
wwoodsif you want/need access to it, let me know11:16
wwoodsI'm going to create directories for each trigger and start putting tests in there as we write them11:16
wwoodsalong with documentation of the test environment for each trigger11:16
wwoodsactually getting the tests hooked into the infrastructure will be something that we'll work out with rel-eng and infra11:17
wwoodsinitially the results will probably just get emailed to various places11:17
wwoodsbut, again, I'd like to see a unified library that provides each test with different ways to report status - email, sending dbus / amqp messages, etc11:18
wwoodsNow, one of the triggers.. which might not be on that list at the moment11:19
wwoodsbut anyway - post-install tests. because, yes, we plan to finally do automated installs of rawhide daily11:19
wwoodsand have a rawhide status page11:20
wwoodsthis will be built on the lab-in-a-box stuff that jlaska's been working on11:20
-!- wwoods changed the topic of #fedora-meeting to: Fedora QA | FUDCon Recap | lab-in-a-box11:20
wwoodsjlaska: do you have time to give a quick summary of that stuff? or a link?11:20
jds2001so there's stuff at jlaska.livejournal.com on how to set it up.11:22
jlaskasure ... sorry distracted11:22
wwoodswell, that's OK, I can try to fake it11:22
jds2001take it away jlaska :)11:22
jlaskajds2001: thanks, yeah, I'm posting my experiences in jlaska.livejournal.com11:22
jlaskaI'll craft up a post tomorrow on how to use SNAKE kickstart templates within the the cobbler env11:22
-!- mxcarron [n=maxime@fedora/Pingoomax] has quit [Read error: 104 (Connection reset by peer)]11:23
-!- mxcarron [n=maxime@fedora/Pingoomax] has joined #fedora-meeting11:24
wwoodsthe basic lab-in-a-box concept is putting together a set of software (cobbler, koan, snake) to allow really, really easy test-installation of Fedora releases11:24
jlaskaI've been discussing with some folks in the last few days to determine the best mechanism to "package" this11:24
wwoodsin a "lab" as small as two systems with a crossover cable, or one system with virtualization capabilities11:24
jlaskafor my purposes, this is all on one system, but uses the same utilities as beaker and can support additional hardware given certain criteria are met11:25
wwoodsremote power control? serial console muxes?11:27
f13sorry, here now.11:27
f13got a little late start this morning.11:27
jlaskaright right, and I can start to craft up more blog/howto's for stuff like that11:27
f13jlaska: I haven't read through, but do you have details on being able to throw arbitrary trees at cobbler, like say test composes each day?11:27
jlaskaf13: that's something that we quickly hacked up at the lunch'n'learn11:28
f13and how to purge off cobbler's view of those trees so that when you purge the tree, cobbler doesn't get upset?11:28
jlaskaI'll get details on that posted soon11:28
-!- MooDoo [n=paulmell@cpc2-mapp2-0-0-cust80.nott.cable.ntl.com] has left #fedora-meeting []11:28
jlaskabpeck has some helpers for that ... I need to talk w/ mpdehaan hopefully on Friday to determine some packaging issues (whether to create a new tool, or push stuff into other existing projects)11:29
wwoodsso this will let us do automatic rawhide installs - very simple installs at first, progressing to more complex tests as time/hardware capablity allows11:30
wwoodsand should also allow us to do more automated install testing on releases11:30
wwoodsanything else we need to cover on lab-in-a-box?11:31
jlaskathat's all I've got right now, I'll likely have more updates next week11:31
wwoodscool. look forward to hearing about it!11:32
wwoodsthe next thing on my list is automated / simplified problem reporting11:33
wwoodswhich is kind of a mouthful11:33
-!- wwoods changed the topic of #fedora-meeting to: Fedora QA | FUDCon Recap | oops.fedoraproject.org11:33
wwoodslet's call it oops.fedoraproject.org for now11:33
wwoodsthe concept is this: we want to set up a system similar to kerneloops11:34
wwoodsin that it allows automated and anonymous filing of problem reports11:34
wwoodsbut for everything in Fedora11:35
wwoodswe'll gather aggregate data from this Big Oops Database and cherry-pick critical, common, or easy-to-fix problems and push them to Bugzilla11:35
jds2001my guess is that most of it needs to go upstream.11:36
wwoodswe'll have a simple "Report a problem" app for people to run on their systems11:36
wwoodsyeah - there are chunks of this that are philisophically very similar to Apport11:36
wwoodsand I'm thinking about giving in and extending the Apport reporting format to handle the things we want to do11:37
wwoodsand/or using chunks of their UI11:37
-!- mxcarron [n=maxime@fedora/Pingoomax] has quit [Read error: 104 (Connection reset by peer)]11:37
wwoodsthey're trying to get that to be a distro-agnostic tool and it seems silly to reinvent the wheel 11:37
wwoodsbut at the same time.. last I checked there were some questionable design decisions11:38
jds2001yeah, i can't claim to have looked much at apport.11:38
wwoodsthe definition of "upstream" is unclear here, too11:39
wwoodssince this is a tool for users to report bugs to the *distributor* of their software11:39
wwoodsnot the *authors*11:39
wwoodsfor example, enterprise customers will likely want to run their own oops.helpdesk.globocorp.biz server11:40
-!- alexxed [n=alex@dyn-] has joined #fedora-meeting11:40
wwoodsso their IT department can filter stuff and report it to Fedora - or Red Hat - as appropriate11:40
wwoodsso it doesn't make sense to try to push this into GNOME, or bug-buddy11:41
wwoodsbut it benefits all Linux distros to have a tool like this11:41
jds2001right, i just mean that a lot of the data is going to have to go upstream (GNOME) at some point, but the Fedora/RHEL maintainers are in the best posistion to do that.11:41
wwoodsoh! you mean most of the *data* needs to go upstream11:42
wwoodsyes, that's absolutely correct - in a lot of cases we'll want to push the data from oops.fp.o up to GNOME 11:42
* wwoods brb11:45
-!- QuickStart [n=QUICKSTA@pool-71-172-136-28.nwrknj.east.verizon.net] has joined #fedora-meeting11:47
wwoodssorry about that11:47
wwoodsjds2001: yeah, I misunderstood - I thought you were saying this code needed to go upstream11:47
wwoodsbut yeah, we'll definitely want to be able to push collected data upstream to GNOME / kernel / freedesktop / etc.11:47
wwoodsso this does involve writing a server-side to collect data11:48
wwoodsa webUI to view the data11:48
-!- openpercept [n=openperc@unaffiliated/openpercept] has quit [Read error: 104 (Connection reset by peer)]11:49
wwoodsclient tools to catch crashes or just allow manual problem submission11:49
jds2001ideally both automatic and manual.11:49
wwoodsplus other stuff to, say, collect debuginfo to get proper tracebacks before submission11:49
wwoodserr, yes, we want both automated and manual reporting11:50
-!- J5 [n=quintice@c-66-31-41-146.hsd1.ma.comcast.net] has joined #fedora-meeting11:50
jds2001so ubuntu does that retracing serverside.11:50
wwoodsright, which is a horrible security problem, since IIRC they upload the core file for later retracing11:50
jds2001right, noticed that - have no idea how they get away with that.11:51
f13users likely just don't know it's happening11:51
f13(and ubuntu just doesn't care)11:51
jds2001"hi, let me give you my bank account number!"11:51
wwoodsit's opt-in, and most people aren't aware of what info is/could be in there11:51
wwoodsoh plus they're not anonymous, AFAIK11:51
wwoodsyou need a launchpad cookie for it to work11:52
wwoodsat least that was the case the last time I looked11:52
wwoodsso that data is directly connected to your email address / launchpad identity11:52
wwoodswe won't be doing that.11:52
wwoodsso, a further step is to have default menu items / widgets to allow this from inside any app that wants to support it11:53
wwoodsso you get unified problem reporting for, say, anaconda, setroubleshoot, GNOME, KDE, etc.11:54
f13or at least a library for one11:54
wwoodsGNOME and KDE are obviously harder problems but putting it into the apps that are RH upstream should be a good idea11:54
wwoodslibrary / glade files / etc.11:54
wwoodshad some long talks with UI designer folks about what a tool like this should look like11:55
wwoodsso I'd like to have one UI that didn't suck that everyone could use11:55
wwoodsthat would be nice.11:55
wwoodsanyway. that's a much longer-term project than the autoqa stuff11:55
wwoodsbut it's something I talked about a lot.11:55
-!- wwoods changed the topic of #fedora-meeting to: Fedora QA | FUDCon Recap | Bodhi changes11:56
wwoodsanother thing we discussed a couple of times: bodhi changes11:56
wwoodsThe sound-bite version is: The updates system should be user-driven, not developer-driven11:56
-!- mxcarron [n=maxime@fedora/Pingoomax] has joined #fedora-meeting11:57
wwoodsAnd we should pay closer attention to some packages than others11:57
wwoodsbasically we want to make some changes like.. instead of nagging developers when a package they built sits in bodhi11:57
-!- J5 [n=quintice@c-66-31-41-146.hsd1.ma.comcast.net] has quit ["Ex-Chat"]11:58
wwoodswe *should* be sending mail to the reporters/cc lists of all the bugs listed11:58
-!- J5 [n=quintice@c-66-31-41-146.hsd1.ma.comcast.net] has joined #fedora-meeting11:58
wwoodsif nobody cares enough to test the fix for the problem, there's no point in destabilizing things by nagging the maintainer to push out an untested package11:58
wwoodsnow, requiring more user/tester help to get updates out means that we may need to start tracking karma etc. for testers11:59
-!- Matias_Arg [n=root@host14.190-31-208.telecom.net.ar] has joined #fedora-meeting12:00
jds2001we already do that , because the fact that it got pushed to testing is a comment in the bug.12:00
-!- Matias_Arg [n=root@host14.190-31-208.telecom.net.ar] has left #fedora-meeting []12:00
-!- giallu [n=giallu@fedora/giallu] has quit [Read error: 110 (Connection timed out)]12:00
wwoodsjds2001: true, but the wording may need to be changed to indicate the intent12:00
wwoods"this package will not be released until it is confirmed to fix this bug. Please test this package and confirm the fix here:"12:00
wwoodsnow, obviously, this requires easier access to updates-testing12:01
wwoodsideally we want a simple PK-based tool that lets you plug in something like a Fedora update ID12:01
wwoodsand will automatically: enable updates-testing, install that package and its dependencies, and remind you about it in a day or two12:02
wwoods(and/or let you revert to the pre-test version, if that update is broken)12:03
jds2001reverting could be a pain, but I guess we'll have to figure out how to do it.12:04
wwoodsthat's something we're going to try to wrok out with hughsie. A lot of the backend bits are already in bodhi-client12:04
wwoodsI think it's possible, but I'll need to discuss with skvidal and folks12:05
wwoodsanyway, this is another longterm project12:06
wwoodsautoqa takes priority12:06
f13and above that, alpha takes priority12:07
wwoods..is there anything else from FUDCon that I've missed?12:07
wwoodsf13: indeed12:07
wwoodsand I still have to fix preupgrade. boo.12:07
wwoodsreally need to find a co-maintainer (or hand that off to the anaconda team)12:07
wwoodsSo I think that's everything from FUDCon. There's a video of the QA talk: http://alt.fedoraproject.org/pub/alt/videos/2009/FUDConF11/qa.ogg 12:08
f13rawhide is in a bit of a ...state... right now.  I'm working as hard as I can to identify the problems, and poke anaconda guys (pjones is who to bug right now for rawhide issues) to help debug and produce fixes12:08
wwoodsand now, Alpha.12:09
f13I /really/ hope to have a installable with kickstart rawhide by tomorrow12:09
-!- wwoods changed the topic of #fedora-meeting to: Fedora QA | Rawhide (f11alpha) status12:09
wwoodsf13: anything you need help (re)testing?12:10
-!- alexxed [n=alex@dyn-] has quit [Read error: 104 (Connection reset by peer)]12:10
-!- openpercept [n=openperc@unaffiliated/openpercept] has joined #fedora-meeting12:10
-!- alexxed [n=alex@dyn-] has joined #fedora-meeting12:11
f13wwoods: not yet.  Current rawhide will bring up network in loader, if you retry after the first timeout.12:11
f13I think that's a dbus issue there, but need to get with walters and dcbw to debug that part12:11
f13next you'll need an updates.img from clumens if you're using kickstart as it has a parsing bug: http://clumens.fedorapeople.org/updates.img12:11
f13after that I'm running into not being able to see my virt disk (virtio being used here) so I may have to move on to real hardware.12:12
f13we need to get a wiki page up with the testing targets, so that we can do some pre-testing to see where we're broken.12:14
-!- mxcarron [n=maxime@fedora/Pingoomax] has quit [Read error: 104 (Connection reset by peer)]12:14
wwoodshm, okay. I guess I'll just carry over the old F10 final test list12:16
wwoodsmaybe drop all that wacky crypt stuff 12:16
wwoodsand add one or two simple crypt cases12:16
f13yeah, for alpha I want to be able to get a basic install done12:17
-!- spoleeba [n=one@fedora/Jef] has joined #fedora-meeting12:19
wwoodsafter that, we'll see what we can do12:19
wwoodsokay. anything else we need to discuss?12:20
-!- linuxguru [n=linuxgur@unaffiliated/linuxguru] has quit [Read error: 110 (Connection timed out)]12:21
wwoodsalright then. meeting over, thanks for your time, folks12:21
wwoodslog will be up shortly12:21
--- Log closed Wed Jan 14 12:21:51 2009

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!