Open in new window / Try shogun cloud
--- Log opened Fri Oct 25 00:00:56 2013
shogun-buildbotbuild #80 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at  blamelist: Fernando Iglesias <>00:11
-!- new_lido [~walid@] has quit [Quit: Leaving]00:22
-!- shogun-notifier- [] has quit [Quit: transmission timeout]00:34
-!- iglesiasg [] has quit [Quit: Leaving]00:45
shogun-buildbotbuild #99 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at  blamelist: Fernando Iglesias <>00:53
-!- FSCV [~FSCV@] has quit [Quit: Leaving]01:13
-!- zxtx [] has quit [Ping timeout: 268 seconds]04:05
-!- zxtx [] has joined #shogun05:18
-!- sonne|osx [] has joined #shogun05:25
@sonney2kshogun-buildbot, force build --branch=develop precise06:36
shogun-buildbotno such builder 'precise'06:36
@sonney2kshogun-buildbot, force build --branch=develop 'precise - libshogun'06:36
shogun-buildbotbuild forced [ETA 19m47s]06:36
shogun-buildbotI'll give a shout when the build finishes06:36
-!- HeikoS [] has joined #shogun06:40
-!- mode/#shogun [+o HeikoS] by ChanServ06:40
@HeikoSr?ln s?lm06:40
shogun-buildbotbuild #245 of precise - libshogun is complete: Success [build successful]  Build details are at
-!- besser82 [] has joined #shogun06:56
-!- besser82 [] has quit [Changing host]06:56
-!- besser82 [~besser82@fedora/besser82] has joined #shogun06:56
@wikingwe have docker image for shogun07:43
@wikingbut this is a simple shogun image07:44
@wikingi'll make soon the python modular image as well07:44
@HeikoSwhooo :)07:53
@sonney2kwiking, nice :)07:53
-!- HeikoS [] has quit [Quit: Leaving.]08:04
@wikingjust try this08:09
@wikingsudo docker pull vigsterkr/shogun08:09
@wikingand i thought that this only distributes the Dockerfile, i.e. how to create the image...08:10
@wikingbut not08:10
@wikingit is actually distributing the fs08:10
@wikingso the binaries...08:10
@wikinglet's hope today we get the fedora stuff and that it runs the latest open stack08:10
@wikingbecause then we can play with the new docker images :P08:10
@wikingi'll have to keep updating both of the images...08:12
@wikingi suppose it'll be done by a nightly script08:12
-!- shogun-notifier- [] has joined #shogun08:15
shogun-notifier-shogun: Soeren Sonnenburg :develop * 2fb5664 / src/interfaces/modular/SGBase.i:
shogun-notifier-shogun: fix hdf5 / ascii pickle deserialization08:15
@wikingbesser82: ping08:43
besser82wiking: pong  :D08:44
@wikingbesser82: any news?08:45
besser82wiking: storage for cloud-appliance is nearly finshed  ;)08:45
@wikingmeaning? :)08:46
@wikingor ETA?08:46
@sonney2kwiking, well rather one per *final* release :)08:47
@sonney2kwiking, except for the demos we are in release shape now08:48
@sonney2kwiking, if you can do anything about matlab / the usr/local install'd issue that would be good08:48
besser82wiking: just need to format and mount the storage....08:49
besser82wiking: any preference?  Filesystem? Mountpoint?08:50
@wikingsonney2k: well i though to have shogun:latest (that is following the develop repo)08:51
@wikingsonney2k: and i thought like shogun:<release version>08:51
@wikingso people can choose ;P08:52
@wikingsonney2k: docker supports tagging... so that's a good thing ;)08:52
@wikingbesser82: simple ext4 should be fine08:52
@wikingbesser82: or is this a ramfs?08:52
besser82wiking: but running version of openStack, doesn't yet  :(08:52
besser82wiking: no, real hdd storage08:53
@wikingbesser82: ah then simple ext4 should be fine08:53
@wikingand just mount it for /mnt08:53
@wikingbesser82: you mean the running version of openstack doesn't support docker?08:53
@wikingbesser82: it's not that big a deal...08:53
besser82wiking: ahh, ok  :)08:54
besser82wiking: are you sure with mounting in /mnt?08:54
@wikingbesser82: can u copy paste `uname -a` ?08:54
@wikingbesser82: where else?08:54
besser82wiking: I'd propose /srv/$somewhat08:54
@wikingbesser82: fine...08:54
besser82wiking: by meanings of SElinux contexts that would be safer08:54
@wikingit's good for me basically08:55
@wikinganything would be fine ;)08:55
@wikingjust let's have it running already08:55
besser82wiking: kernel on the machine is 3.11.X08:55
@wikingbesser82: ah ok then we can haz docker by our own08:55
@wikingit needs 3.808:55
@wikingbut just put it together and let's see if we can ssh ;08:56
besser82wiking: 3.8 is old-school by fedora means  :-P08:56
besser82wiking: no even in officail repos anymore for F18+  ;)08:56
besser82wiking: so, I'll setup c-a after my next coffee && cig08:57
besser82sonney2k: I'm on that problems with cmake, but unfortunately some stuff is _really_ problematic, e.g. correctly building the SWIG'ed java-bindings08:58
@wikingbesser82: why is that a problem?08:58
besser82sonney2k: Actually it fails for me by any meanings08:59
besser82wiking: ^^^08:59
@wikingi mean it worksforme status now :P08:59
@wikingso that's why i wonder what's your issue with java modular08:59
besser82wiking: at first javac complains about not finding ....*.java08:59
besser82wiking: got that fixed then08:59
besser82wiking: then it fails on building Loader.class09:00
@wikingbesser82: u r talking now about your new cmake scripts or with current cmake files?09:00
besser82wiking: no package org.shogun...09:00
besser82wiking: current09:00
@wikingbesser82: and which java r u trying to use09:00
besser82wiking: openJDK 1.7.009:00
@wikingoracle java or some opejdk09:00
@wikingwell i guess you have your answer ;)09:00
besser82wiking: WTF?!?09:01
besser82wiking: Who used the oracle sh*t?09:01
@wikingopenjdk is really not the best shit for java09:01
@wikingwell everybody who uses java09:01
@wikingi mean openjdk is a nice dream09:01
@wikingbut it's really bad09:01
@wikingregardless of shogun09:01
-!- sonne|osx [] has quit [Quit: sonne|osx]09:01
besser82wiking: even hadoop, mesos && mahout are running fine with it09:02
@wikingbesser82: i had problem both with mahout and solr09:02
besser82wiking: no, problems here on the redhat/fedora-front  ;)09:03
@wikingbesser82: that's unrelated of the distro09:04
besser82wiking: I know, but we actually got it working  ;)09:05
besser82wiking: by no means ready for production, but it runs...09:05
besser82wiking, sonney2k: but if you wouldn't mind, I can get some folks from the fedora-java-sig to make it running  ;)09:08
shogun-buildbotbuild #80 of clang34 - undefined behaviour analysis is complete: Failure [failed test]  Build details are at  blamelist: Soeren Sonnenburg <>09:19
@wikingbesser82: sure... get the c-a runnning ;)09:25
* wiking off 4 a c&c break as well09:25
shogun-buildbotbuild #81 of clang34 - thread analysis is complete: Failure [failed test]  Build details are at  blamelist: Soeren Sonnenburg <>09:33
-!- lisitsyn1 [~lisitsyn@] has quit [Quit: Leaving.]09:54
-!- new_lido [~walid@] has joined #shogun10:03
shogun-buildbotbuild #100 of clang34 - static analysis is complete: Failure [failed analyse]  Build details are at  blamelist: Soeren Sonnenburg <>10:15
@wikingbesser82: y10:21
besser82wiking: wassup?10:24
@wikingbesser82: mkfs.ext4 status?10:28
besser82running  ;)10:34
@wikingssh ip? :)10:35
besser82will setup in few minutes...10:39
@wikingok w8ing10:43
@wikingi'm finalizing the docker image for shogun + ipython10:43
-!- new_lido [~walid@] has quit [Ping timeout: 260 seconds]10:55
-!- new_lido [~walid@] has joined #shogun11:07
-!- shogun-notifier- [] has quit [Quit: transmission timeout]11:15
@wikingok shogun + ipython docker image is done11:15
@wikingi'm just pushing it atm11:15
@wikingmmm i think there's still a problem with it but i'll fix after commit11:17
-!- new_lido [~walid@] has quit [Quit: Leaving]11:39
@wikingbesser82: ping?12:03
-!- lisitsyn [~lisitsyn@] has joined #shogun12:23
-!- new_lido [~walid@] has joined #shogun12:52
-!- iglesiasg [~iglesias@2001:6b0:1:1da0:b854:c0fd:3503:f6c] has joined #shogun13:02
-!- mode/#shogun [+o iglesiasg] by ChanServ13:02
-!- new_lido [~walid@] has quit [Read error: Operation timed out]14:17
@wikingbesser82: no news?14:17
-!- new_lido [~walid@] has joined #shogun14:29
@wikingoookeey so instructions has been sent to the mailing list14:35
-!- new_lido [~walid@] has quit [Ping timeout: 252 seconds]14:37
@iglesiasgwiking, hey! I just read the mail. It sounds pretty awesome. I will spend some time with it during the weekend14:38
@iglesiasglooking forward to it14:38
@wikingiglesiasg: it's pretty cool14:38
@wikingno need for compilation14:38
@wikingjust get the docker binary working and then it's running out of box14:38
@wikingand it's really cool that they provide a repository for sharing images ;P14:39
@iglesiasgnada haha14:39
@wikingiglesiasg: we'll use this for the cloud arch14:39
@wikingas this is a pretty good way of jailing an ipython notebook server14:40
@wikingfor each ipython session a new docker container will fired up14:40
@wikingand then basically when the session is over the container is destroyed ;P14:40
@wikingthe good thing is that this doesn't involve any VM stuff... i.e. it runs natively on the system14:41
@wikingof course there's no native client for it atm on OSX14:41
@wikingbut for that they prepared a vagrant image14:41
@wikingbut you should be able to have it on archlinux14:42
@wikingonly thing is that u have to take care of the right kernel settings14:42
-!- new_lido [~walid@] has joined #shogun14:49
-!- iglesiasg [~iglesias@2001:6b0:1:1da0:b854:c0fd:3503:f6c] has left #shogun ["Ex-Chat"]14:50
-!- iglesiasg [~iglesias@2001:6b0:1:1da0:b854:c0fd:3503:f6c] has joined #shogun14:50
-!- mode/#shogun [+o iglesiasg] by ChanServ14:50
-!- parijat [~parijat@] has joined #shogun14:54
parijathi, I wanted to know that if I make changes in the shogun cpp files, how can I put these changes into effect? do i have to run cmake and make in the root directory again?14:57
thoralfparijat: Just recompile everything and run the unit tests (in case you broke something).14:59
-!- new_lido [~walid@] has quit [Ping timeout: 252 seconds]15:00
-!- parijat_ [~parijat@] has joined #shogun15:02
-!- parijat_ [~parijat@] has quit [Client Quit]15:03
-!- parijat_ [~parijat@] has joined #shogun15:03
thoralfwiking: Comments to you're email: Why do I have to run ipython as root?15:05
-!- parijat [~parijat@] has quit [Ping timeout: 248 seconds]15:05
parijat_thanks @thoralf. Just to make sure recompile everything means to run cmake and then make in the root directory (ie shogun-develop) right?)15:05
thoralfparijat_: Yes.  "make -j<#processes> (all|shogun-unit-test|unit-tests|install)"15:08
parijat_ok thanks. let me try.15:11
-!- parijat_ [~parijat@] has quit [Quit: Leaving]15:12
@wikingthoralf: doh?15:13
@wikingthoralf: i mean it's chrooted15:13
@wikingthoralf: so it's not that big of a deal15:14
thoralfwiking: That's true, but if I'm trying to play around on a system on which I'm not sudoer?15:15
@wikingthoralf: aaah wait... you are now talking about docker commands?15:16
thoralfwiking: Sorry, missing context.  Yes. ;)15:16
@wikingthoralf: that's out of my scope... talk with docker developers15:16
@wikingwhy do u need sudo15:16
@wikingbut imo it's all about the kernel apis it's using15:16
@wikingand that they are not available by default for simple users15:16
@wikingbut then again i'm not sooo familiar with docker internals15:17
@wikingi'm just following the official docker documentation15:17
@wikingwhich uses sudo everywhere for docker15:17
@wikingif u wanna know more: /join #docker or write to mailing list15:18
thoralfYeah. ;)15:19
@wikingthere u go... it depends on your docker setup... but then again by default it shoudl be sudo... :P15:21
@wikingthoralf: "Just to make sure recompile everything means to run cmake and then make in the root directory (ie shogun-develop) right?)"15:56
@wikingthat was actually wrong...15:57
@wikingnot in the root but in the build...15:57
thoralfroot directory (ie shogun-develop) <-- I thought shogun-develop is his build dir.15:57
thoralfNot clear.15:57
thoralfBtw. rebuilding everything shouldn't be neccessary any more, right?15:58
@wikinghehe yeh i hope so :D15:58
-!- shogun-notifier- [] has joined #shogun16:27
shogun-notifier-shogun-web: Soeren Sonnenburg :master * cac1ff4 / templates/home.html:
shogun-notifier-shogun-web: link screenshots16:27
@sonney2kalright guys16:28
@sonney2kiglesiasg, wiking, lisitsyn, thoralf besser82 - release sprint!16:28
@iglesiasglet's do it16:29
lisitsynsonney2k: good catch I am more or less available16:29
lisitsynthough have a bus in 2 hrs16:29
lisitsyniglesiasg: how can we make that notebook16:30
-!- FSCV [~FSCV@] has joined #shogun16:30
@iglesiasglisitsyn, do you have any idea about it written down so far?16:30
lisitsynno I was a bit stucked16:30
@wikingbesser82: ping ping ping16:31
@sonney2kheiko is writing long blog posts currently16:31
@sonney2kbuildbots are all green16:31
@sonney2kand I will do demos16:31
@wikingsonney2k: no they are not16:31
@wikingsonney2k: there's at least one major concurency error16:31
lisitsyniglesiasg: lets join some google doc then16:31
@iglesiasglisitsyn, sure16:32
@sonney2kwiking, well next release we can do better then16:32
@iglesiasglisitsyn, I have time to work on this this evening after dinner and during the weekend16:32
@wikingsonney2k: you can run the demos in docker image16:32
lisitsyniglesiasg: uhmm I guess sonney2k wants it asap :D16:32
@sonney2kwiking, well they are running already but we can switch later16:32
lisitsyniglesiasg: when is your dinner?16:32
@wikingsonney2k: where r they running atm?16:33
@iglesiasglisitsyn, long to come still16:33
@iglesiasglisitsyn, like in about 4-5 hours16:33
@wikingok so fatbot16:33
@sonney2kso if you see sth to polish please do it16:33
@sonney2kI mean documentation16:33
lisitsynokay I'll start and pass it over to you16:33
@wikingsonney2k: there's heaps of issues still..16:33
@iglesiasglisitsyn, sounds good!16:33
@sonney2kwiking, which?16:34
@wikingfirst 4 could be closed by 1 fix16:34
@sonney2kwiking, well we won't have workshop videos16:34
@sonney2kso what lets move them to 3.116:34
lisitsyniglesiasg: I think I want to cover just 5 methods16:35
@wikingsonney2k: what's the sudden rush?16:35
@iglesiasglisitsyn, we can get the theory from tapkee's website16:35
@sonney2knothing release critical16:35
lisitsynyes iglesiasg16:35
lisitsynsonney2k: agree, these data races are not major16:35
@wikingsonney2k: we have a thread sync error, and that's not critical?16:35
@sonney2kwiking, no16:35
@wikingwho's a data race not critical?16:35
@sonney2kit is just one bug among many16:35
@iglesiasglisitsyn, so probably there is not much cost including many methods16:36
lisitsynwiking: I think it is not16:36
@wikinglisitsyn: and how is it not a critical?16:36
@iglesiasglisitsyn, so we could present a couple of figures with embeddings16:36
@sonney2kand it is 'minor'16:36
lisitsynwiking: no I get your message but we have other bugs as well16:36
lisitsynwe won't release any day if we approach to fix everything16:36
lisitsynjust lower your threshold ;)16:36
@wikinglisitsyn: not eveyrthing we've moved quite some stuff already to 3.116:36
@iglesiasglisitsyn, any idea how could we compare the results of different methods applied to the same data set?16:37
lisitsyniglesiasg: not yet16:37
@sonney2kwiking, if you have time fix as much as you can and then we will release with that16:37
@iglesiasglisitsyn, maybe for some labelled data set we apply knn in the original space16:37
@iglesiasglisitsyn, and then the same using the data after dim red16:37
@sonney2kwiking, we just have to release *very soon* or we will never16:37
@iglesiasglisitsyn, and we make a table or a bar plot of classification accuracy using different dimred methods and that's pretty much it. What do you think?16:38
@wikingsonney2k: ok then at least give this weekend16:38
lisitsyniglesiasg: sounds good but no idea what the outcome would look like16:38
@iglesiasghaha me neither16:38
@sonney2kwiking, we will prepare everything today - if you can fix (the critical cmake erros) ws videos - that is much more important than some function crashing sometimes16:40
-!- sonne|osx [] has joined #shogun16:40
@iglesiasgsonney2k, so release today?!16:41
@sonney2kiglesiasg, I am on the plane with heiko today so we try to get everything blog / release post like in shape16:42
@wikingsonney2k: can we have the release on monday?16:42
@sonney2kiglesiasg, and I will write some shell script to do the release tgz16:42
@sonney2kwiking, what exactly do you still want to do / fix?16:42
@sonney2kwiking, cmake with matlab? cmake with /usr/local/* or ws videos?16:42
@wikingand other static interfaces16:42
@sonney2kwiking, I mean why delay? these issues are open for months16:43
@wikingsonney2k: yeas16:43
@wikingbut this weekend i have time16:43
@wikingunlike other weekends...16:43
@wikingi mean i'm not fulltime working on shogun (not like during summer)16:43
@sonney2kwiking, ok then we will prepare everything then so we just have to push a button16:43
@sonney2kwiking, fix what you can / want to16:44
@sonney2kwiking, but we *will* release monday latest16:44
@wikingok monday 12:00 UTC16:44
lisitsynmonday would be better for me cause I can prepare better notebook16:44
@wikingi'll have things ready by sunday night16:44
@wikingor at least as much as i can16:44
-!- new_lido [~walid@] has joined #shogun16:47
-!- new_lido [~walid@] has quit [Ping timeout: 241 seconds]16:57
@sonney2klisitsyn, well the sooner the better17:08
-!- new_lido [~walid@] has joined #shogun17:09
-!- sonne|osx [] has quit [Quit: sonne|osx]17:10
-!- iglesiasg [~iglesias@2001:6b0:1:1da0:b854:c0fd:3503:f6c] has quit [Quit: Ex-Chat]17:23
-!- new_lido [~walid@] has quit [Ping timeout: 252 seconds]17:37
-!- lisitsyn [~lisitsyn@] has quit [Quit: Leaving.]18:06
-!- zxtx [] has quit [Ping timeout: 260 seconds]18:26
-!- shogun-notifier- [] has quit [Quit: transmission timeout]19:26
-!- new_lido [~walid@] has joined #shogun20:41
-!- new_lido [~walid@] has quit [Quit: Leaving]20:54
-!- thoralf [] has quit [Quit: Konversation terminated!]20:56
-!- new_lido [~walid@] has joined #shogun20:56
-!- iglesiasg [] has joined #shogun21:46
-!- mode/#shogun [+o iglesiasg] by ChanServ21:46
-!- lisitsyn [] has joined #shogun22:00
@iglesiasglisitsyn, hey22:40
@iglesiasglisitsyn, what's up22:40
lisitsyniglesiasg: hey22:40
lisitsyniglesiasg: I haven't managed to write anything interesting yet :)22:40
@iglesiasglisitsyn, no problem. I was thinking about starting an script to check this knn after dim red idea22:41
lisitsynoh would be nice!22:41
@iglesiasglisitsyn, are you ok with it?22:41
lisitsynyes totally22:41
@iglesiasgany idea for the data set to use?22:41
@iglesiasgif not, I will take this portion of MNIST I used for the LMNN notebook22:41
@iglesiasgor I could go with the metagenomics data set22:42
@iglesiasgsort of high dimensional, so it can be interesting in this case22:42
lisitsyniglesiasg: yeah metagenomics one sounds more interesting22:44
@iglesiasglisitsyn, 85 examples with 1472 features, 3 classes22:46
@iglesiasgthere is the result applying td-sne22:46
lisitsyniglesiasg: pretty big stuff23:14
--- Log closed Sat Oct 26 00:00:57 2013