Open in new window / Try shogun cloud
--- Log opened Mon Jun 24 00:00:02 2013
-!- lisitsyn [] has left #shogun []00:11
-!- van51 [] has quit [Quit: Leaving.]01:23
-!- nube [~rho@] has quit [Ping timeout: 246 seconds]02:05
-!- nube [~rho@] has joined #shogun02:19
-!- foulwall` [~user@] has joined #shogun02:54
shogun-buildbotbuild #437 of nightly_default is complete: Failure [failed compile]  Build details are at
-!- hushell1 [~hushell@] has quit [Ping timeout: 264 seconds]03:19
-!- nube [~rho@] has quit [Quit: Leaving.]03:46
-!- foulwall` [~user@] has quit [Ping timeout: 268 seconds]04:38
-!- FSCV [~FSCV@] has quit [Quit: Leaving]05:18
-!- hushell1 [] has joined #shogun05:28
-!- hushell2 [] has joined #shogun05:44
-!- hushell1 [] has quit [Ping timeout: 246 seconds]05:45
-!- hushell2 is now known as hushell07:26
-!- nube [~rho@] has joined #shogun07:30
-!- sonne|work [] has joined #shogun08:01
hushellsonney2k: Hello? Can SG_UNREF(var) called if var is NULL?09:08
sonne|workhushell: yes. it will even NULL var afterwards09:10
sonne|work(makes only sense if it was not of course)09:10
hushellsonne|work: Thanks! I just checked the doc: if (x) { if ((x)->unref()==0) (x)=NULL;09:11
hushellI should do this before :)09:12
-!- votjakovr [] has joined #shogun09:43
-!- nube [~rho@] has quit [Ping timeout: 240 seconds]10:19
-!- nube [~rho@] has joined #shogun10:44
-!- vgorbati [~vgorbati@] has joined #shogun10:48
-!- vgorbati [~vgorbati@] has quit [Quit: vgorbati]11:09
-!- iglesiasg [] has joined #shogun11:26
-!- mode/#shogun [+o iglesiasg] by ChanServ11:26
-!- gsomix [~Miranda@] has joined #shogun11:27
gsomixone more exam is passed11:27
gsomixso, there is only one now11:28
gsomixsonne|work: sonney2k hey, glad to see you. :)11:28
sonne|workgsomix: when is you last one?11:33
gsomixsonne|work: 27 June, state exam11:33
gsomixand then full-time job at shogun11:34
sonne|workgsomix: so until then you have no time I guess?11:35
gsomixsonne|work: yep :(11:38
sonne|workgsomix: then good luck but don't forget you weekly report (today!) though!11:40
-!- hushell [] has quit [Quit: WeeChat 0.3.7]11:42
gsomixsonne|work: of course11:50
@iglesiasgC++ question12:01
@iglesiasgwhen does it actually matter to use a const return value?12:01
-!- van51 [] has joined #shogun12:07
sonne|workhey van51!12:08
sonne|workhow is it going?12:08
van51sonne|work: hey!12:09
van51sonne|work: it's going ok :)12:09
sonne|workohh and I hope you can do the language text example too. for the image one we already have some matlab code that porting won't need much time12:09
van51sonne|work: I wanted to ask you12:09
van51sonne|work: you mean the language detection?12:09
van51sonne|work: are you going to remove the SGString ?12:10
van51sonne|work: and the StringFeatures?12:10
sonne|workthat should be rather easy to do. just n-grams and fetch ascii based languages from wikipedia for a few thousand articles12:11
sonne|workyes SGVector it will be12:11
sonne|workStingFeatures will stay of course12:11
van51sonne|work: I was thinking of moving the tokenization to an earlier stage of the pipeline12:12
van51and instead of storing the documents in stringfeatures and having to tokenize them when they are needed12:13
van51I was thinking of storing a list of tokens12:13
van51or a list of SGVectors12:13
van51for each document12:13
van51sonne|work: what do you think?12:14
-!- nube [~rho@] has quit [Quit: Leaving.]12:14
van51also I think that the current way will be easier to be extended to streamed features12:15
van51sorry if I'm being overwhelming :)12:15
-!- lisitsyn [] has joined #shogun12:28
lisitsynsonne|work: hey there12:29
lisitsynvotjakovr: van51: gsomix: pickle27: weekly reports please ;)12:29
@iglesiasgand me too!12:30
lisitsyniglesiasg: oops sorry missed you12:30
@iglesiasghehe no problem12:30
lisitsyniglesiasg: as for your question12:30
@iglesiasglisitsyn: btw enlighten me12:30
@iglesiasgyeah I was going to ask you about it right now :)12:30
lisitsynconst return value is useless12:30
lisitsynI don't see any case12:30
@iglesiasgyou sure?12:31
lisitsynconst Object f();?12:31
lisitsynmakes no sense to me at all12:31
lisitsynconst Object& would be something more interesting12:31
@iglesiasgand const Object* as well I guess12:31
lisitsynthough that's dangerous and should be avoided12:31
lisitsynconst Object* or const Object* const yes12:32
lisitsyniglesiasg: if you return by value just use value type12:32
lisitsynI don't know any case when you restrict new object to be const12:33
lisitsynthat's the user preference to make it const12:33
@iglesiasgthe concepts of return by value and return by reference I understand12:33
@iglesiasgBTW I started wondering because of
@iglesiasgthe two get_features methods there12:34
lisitsyniglesiasg: hmm they are const12:34
lisitsynnot the return value12:34
@iglesiasgthe first return value is const12:35
@iglesiasgcons CFeatures*12:35
lisitsynconst CFeatures*12:35
lisitsynthat's constant pointer12:35
lisitsynlike we can't change where it points to12:35
@iglesiasgpointer to constant rather?12:35
lisitsyniglesiasg: no that's CFeatures* const12:36
@iglesiasgmmm let me test it out12:36
lisitsyniglesiasg: const char* - you can't change the pointer but you can change the string12:37
lisitsynconst char* const12:37
lisitsynyou can't change anything12:37
lisitsynchar* const - you can't change contents but you are free to move the pointer around12:37
@iglesiasgI think it is the other way round12:42
van51how detailed must this weekly report be?12:42
@iglesiasgchar* const you can change pointer but no contents12:42
@iglesiasgvan51: like the ones they have already sent are quite fine I think12:43
van51iglesiasg: I'm almost done, but now that I read it, I'm thinking that someone not familiar with the project will probably not follow :(12:43
@iglesiasgvan51: aham I understand12:44
@iglesiasgvan51: it should be something short that summarizes what you have done12:44
@iglesiasgso for instance Shell's description I think is fine in length12:45
@iglesiasgbut it is difficult to define what the right length is12:45
@iglesiasgit should be something not time consuming at all12:45
@iglesiasgor at least that's the idea :) hehe12:45
@iglesiasgvan51: in any case, if you are done with the one for this week just send it around. Do not worry about improving it or so12:46
lisitsyniglesiasg: yes it is what I said :)12:46
@iglesiasglisitsyn: haha ok, I misunderstood :)12:46
van51ok sent it12:51
van51iglesiasg: thanks :)12:51
van51btw I'm getting an error when I use the ALPHANUM alphabet12:53
van51[ERROR] bool shogun::CAlphabet::check_alphabet(bool): ALPHABET does not contain all symbols in histogram12:53
van51but all I use are letters :/12:53
van51any ideas?12:53
@iglesiasgvan51: what letters are you using?12:54
van51iglesiasg: upper case latin letters12:54
@iglesiasgI have never used that code but I recall from some mail something like only C,A and another letter should be used12:54
@iglesiasgbut I am not sure at all, just recall that from some mail, it could be regarding another class though12:55
van51iglesiasg: I think that's when you use the dna alphabet12:56
@iglesiasgit makes sense12:57
lisitsynvan51: check for \n and etc12:57
van51iglesiasg: I'm probably using something wrong12:57
lisitsynEOL EOF12:57
van51ok solved it, turns out I had a larger length value in the SGString than I should13:09
van51any easy way to have it accept spaces as well now? :)13:09
-!- foulwall|mobile [~foulwall@] has joined #shogun13:10
-!- foulwall|mobile [~foulwall@] has quit [Quit: AndroIRC - Android IRC Client ( )]13:16
-!- gsomix [~Miranda@] has quit [Quit: Miranda IM! Smaller, Faster, Easier.]13:16
sonne|workvan51: if you can make the error message more meaningful please do it13:25
sonne|workvan51: I am jsut not able to. it is ment to say that you have some symbols in your input that are not in the alphabet13:26
sonne|workand if you enable debug you would see the sysmbold13:26
sonne|workvan51: sry was out for lunch.13:27
van51sonne|work: no worries13:27
sonne|workvan51: when you don't have string features it makes IMHO already most sense to directly compute the hashed representation13:27
van51sonne|work: about that error, I fixed it13:27
sonne|workwell a better error message is something you could help with13:28
sonne|workvan51: so just write the converter string -> hashed n gram word features13:28
sonne|workactually it is string -> sparse features with hashes as indices13:29
van51sonne|work: yeah, that is what I have done13:30
van51sonne|work: I'm just wondering what would be better13:31
van51to tokenize the string once in the beginning, or on-the-fly as weel13:31
van51or do you mean already a token by string and not a document?13:31
-!- nube [~rho@] has joined #shogun13:34
sonne|workvan51: it depends what inputs you have: in the online / streaming case it makes sense to tokenize on-the-fly & do the hashing13:43
sonne|workif you have the whole set of strings already then you do the tokenize&hash pass13:43
sonne|workin one go later13:43
sonne|workboth make sense and have their use case13:44
-!- nube [~rho@] has quit [Ping timeout: 246 seconds]13:54
van51sonne|work: yes indeed. well, right now I tokenize and hash on-the-fly when the strings fit in memory13:54
van51should I change it?13:54
sonne|workI didn't git it13:55
sonne|workyou have the whole strings in memory13:55
sonne|workand then you do the tokenize & hashing in one go right?13:55
van51sonne|work: well there is this possibility through the converter13:56
van51but the HashedDocDotFeatures, maintain the documents as a CStringFeatures object13:56
van51and whenever a dot product is needed, is tokenizes and hashes on the fly13:57
sonne|worktotally OK!13:57
sonne|workthe other other option is doing the same via the converter as you said13:57
van51sonne|work: should I make a PR so you can review it as I progress ?14:00
van51or Olivier,Benoit or Quoc14:00
sonne|workvan51: sure! I prefer small snacks anyway14:00
sonne|workso if you have one small part working I would rather merge that already before you do the big thing in one go14:01
van51sonne|work: ok, let me change the commit a bit first then14:02
-!- iglesiasg [] has quit [Ping timeout: 246 seconds]14:04
van51on another note, if I want an alphabet that accepts spaces as well, I have to create a new EAlphabet type?14:06
sonne|workvan51: jsut use RAWBYTE14:07
-!- foulwall [~foulwall@2001:da8:215:503:5466:91ea:7714:9bdb] has joined #shogun14:08
-!- nube [~rho@] has joined #shogun14:08
van51sonne|work: ah cool! thanks14:08
sonne|workhey foulwall!14:14
foulwallhey sonne|work14:15
sonne|workdid you finish your exams?14:15
sonne|workvan51: np14:15
foulwallsonne|work: yes~14:15
sonne|workall good?14:15
foulwallsonne|work: yeah14:15
foulwall1. integrate the  data to the exisiting demos. (regression, clustering)14:16
foulwall2. merge the exisiting binary/multiclass classification demo into the framework.14:16
foulwall3. refractor the javascripts.14:16
foulwall4. find a better way to draw heatmap with contour.14:16
foulwallsonne|work: I planed to do these things this week.14:16
sonne|workfoulwall: excellent plan!14:19
sonne|worklet me know if you have questions / get stuck14:19
sonne|workfoulwall: btw I started setting up a machine for you where you can run the demos...14:20
sonne|worknot sure if internet access is fast for you (machine is in some data centre in berlin)14:20
foulwallsonne|work: wow14:20
sonne|workwell it is just some (not so fast) virtual machine I did rent.14:21
foulwallsonne|work: that's alright.14:21
lisitsynsonne|work: do you want to rent other machine?14:30
lisitsyndigital ocean is pretty low-cost14:30
lisitsyntapkee website runs there14:31
lisitsyn5$ / month14:31
sonne|workno I have this machine for 3 months still and I am planning to cancel the contract past gsoc so all good14:32
-!- zxtx [] has quit [Read error: Operation timed out]14:32
lisitsynsonne|work: alright14:33
-!- thoralf [] has joined #shogun14:50
van51sonne|work: I've made a PR15:01
-!- iglesiasg [~nando@] has joined #shogun15:19
-!- mode/#shogun [+o iglesiasg] by ChanServ15:19
-!- van51 [] has quit [Quit: Leaving.]15:23
-!- zxtx [~zv@] has joined #shogun15:29
-!- zxtx [~zv@] has quit [Ping timeout: 240 seconds]15:51
-!- foulwall [~foulwall@2001:da8:215:503:5466:91ea:7714:9bdb] has quit [Remote host closed the connection]16:01
-!- zxtx [~zv@] has joined #shogun16:12
lisitsynsonne|work: is that expensive to buy a sim-card for a few days in berlin?16:13
thoralflisitsyn: About 10-15 Euro for the card plus 5-10 Euro for a data plan.16:18
thoralflisitsyn: You need it for the workshop in july, right?16:20
lisitsynthoralf: oh thanks16:20
lisitsynthoralf: yes16:20
lisitsynthoralf: just to coordinate with other guys16:20
lisitsynthoralf: what is a data plan?16:20
thoralflisitsyn: Oh, I guessed you need internet access (== data plan ;)).16:21
lisitsynthoralf: ahh I see16:21
lisitsynwell would be good indeed16:22
lisitsynthoralf: not that much! thanks for the info16:22
lisitsynthoralf: may be I can ask you a bit more? ;)16:22
thoralflisitsyn: Yes.16:23
pickle27lisitsyn: I am going to send mine out soon! (trying to finish something up)16:30
lisitsynpickle27: thanks16:32
pickle27is there any way built into shogun to test if a matrix is a permutation matrix?16:50
pickle27iglesiasg: ^16:53
lisitsynpickle27: no I don't know any16:53
pickle27okay just thought I'd check before I coded something16:53
pickle27just finishing up porting the R example to c++, I think it will make a decent unit test16:54
-!- gsomix [~Miranda@] has joined #shogun17:04
-!- zxtx [~zv@] has quit [Read error: Connection reset by peer]17:27
pickle27lisitsyn: did my mail send out? (I feel like my own messages to the list probably don't get sent back to me)17:38
@iglesiasgpickle27: yeah, got it17:38
@iglesiasgnice work btw17:39
pickle27thanks, you too!17:39
@iglesiasgI like the cleaning code aka using Eigen definition :D17:39
pickle27haha yeah, I have this "inner loop" function in C from the R package that I'd like to slowly replace17:39
-!- foulwall [~foulwall@2001:da8:215:c252:59d9:be69:e4d9:cb89] has joined #shogun17:53
lisitsynpickle27: yes your mail is here18:02
lisitsynvotjakovr: now please weekly report ;)18:02
lisitsyngsomix: and you18:02
votjakovrlisitsyn: hi, ah, sure18:04
-!- foulwall [~foulwall@2001:da8:215:c252:59d9:be69:e4d9:cb89] has quit [Remote host closed the connection]18:10
-!- iglesiasg [~nando@] has quit [Ping timeout: 268 seconds]18:21
-!- thoralf [] has quit [Quit: Konversation terminated!]19:18
-!- mode/#shogun [+o sonney2k] by ChanServ19:23
-!- iglesiasg [] has joined #shogun19:29
-!- mode/#shogun [+o iglesiasg] by ChanServ19:29
gsomixlisitsyn: done19:51
votjakovrlisitsyn: me too19:53
lisitsynthanks guys19:54
-!- lisitsyn [] has quit [Quit: Leaving.]19:55
-!- zxtx [~zv@] has joined #shogun20:28
-!- lisitsyn [~lisitsyn@] has joined #shogun20:40
-!- vgorbati [~vgorbati@] has joined #shogun20:50
-!- iglesiasg [] has quit [Ping timeout: 256 seconds]21:39
@sonney2kwiking, new bugs: ../shogun/mathematics/Random.h:86: warning: integer constant is too large for 'long' type21:41
@sonney2kand /usr/lib/gcc/i486-linux-gnu/4.4.5/include/emmintrin.h:32:3: error: #error "SSE2 instruction set not enabled"21:42
@sonney2kIn file included from ../shogun/lib/external/SFMT/SFMT.h:84,21:42
@wikingsonney2k: mmm21:50
@sonney2kwiking, let me try to fix the random +86 one21:51
@sonney2kreturn random_64() & ((uint64_t(-1)<<1)>>1); does the same and makes the compiler happy21:53
@sonney2kwiking, so now we need to fix the SSE2 one still21:53
@sonney2kwiking, ohh and please upgrade json-c on the fbsd machine to v 0.1121:53
@sonney2k0.10 is b0rken21:53
-!- shogun-notifier- [] has joined #shogun21:55
shogun-notifier-shogun: Soeren Sonnenburg :develop * 432634e / src/shogun/mathematics/Random.h:
shogun-notifier-shogun: use equivalent formulation for masking out sign bit in rand functions21:55
@wikingsonney2k: ok... just a sec21:59
@wikingsonney2k: there's no 0.11 json-c port in fbsd ports system22:09
shogun-notifier-shogun: Soeren Sonnenburg :develop * a75ef21 / src/shogun/distance/CustomDistance.cpp,src/shogun/distance/CustomDistance.h:
shogun-notifier-shogun: move template function back to .h to fix compile error22:10
@wikingsonney2k: json-glib would do it?22:10
@wikingjson-glib-0.14.2.tar.xz is possible22:10
@wikingok i see it's not :)22:12
shogun-buildbotbuild #1134 of bsd1 - libshogun is complete: Failure [failed test test_1]  Build details are at  blamelist: Soeren Sonnenburg <>22:12
shogun-buildbotbuild #966 of cyg1 - libshogun is complete: Failure [failed compile]  Build details are at  blamelist: Soeren Sonnenburg <>22:13
shogun-buildbotbuild #1258 of deb3 - modular_interfaces is complete: Failure [failed compile python_modular]  Build details are at  blamelist: Soeren Sonnenburg <>22:14
-!- pickle27 [] has quit [Quit: Leaving]22:14
shogun-buildbotbuild #967 of cyg1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: Soeren Sonnenburg <>22:17
@sonney2kwiking, no json-c :D22:17
shogun-notifier-shogun: Roman Votyakov :develop * 72fa7b7 / / (66 files):
shogun-notifier-shogun: refactor gp framework22:19
shogun-notifier-shogun: Heiko Strathmann :develop * 8ded361 / / (66 files):
shogun-notifier-shogun: Merge pull request #1187 from votjakovr/feature/gp_refactoring22:19
shogun-notifier-shogun: refactor gp framework22:19
@sonney2kwiking, our HAVE_SSE2 test is flawed22:20
@sonney2kwiking, err actually no -msse2 is missing from compile flags22:22
shogun-buildbotbuild #1135 of bsd1 - libshogun is complete: Failure [failed test_1]  Build details are at  blamelist: Soeren Sonnenburg <>22:24
-!- zxtx [~zv@] has quit [Ping timeout: 248 seconds]22:25
-!- iglesiasg [] has joined #shogun22:26
-!- mode/#shogun [+o iglesiasg] by ChanServ22:26
@sonney2kiglesiasg, I had to move the templated customdistance function back to the .h file - all buildbots were failing due to the .h -> .cpp move22:30
-!- lisitsyn [~lisitsyn@] has quit [Ping timeout: 255 seconds]22:33
@sonney2kwiking, do you want to try?22:36
-!- zxtx [~zv@] has joined #shogun22:37
@iglesiasgsonney2k: thank you22:37
@iglesiasgI noticed it gave trouble but preferred to discussed if there was a way to maintain it in the cpp first22:38
@sonney2kiglesiasg, well at this stage it is important to keep the build green22:38
@iglesiasgsonney2k: understood22:39
@sonney2kiglesiasg, there are too many merges coming up22:39
@wikingsonney2k: try what?22:43
@sonney2kwiking, the sse2 issue22:43
@sonney2kI cannot make sense of it22:44
@wikingsonney2k: ah stupid ./configure22:44
@wikingwe should finish cmake :PPP22:44
@sonney2kjust do it then though this is somehting subtle22:44
@sonney2kI figured that gcc -msse2 will define __SSE2__ 122:45
@sonney2kwhich we did not do22:45
@sonney2kbut then even with -msse2 it still doesn't work22:45
@wikingthis is only with gcc i guess22:46
@sonney2kgcc & ooold debian 6.022:46
@sonney2kand i68622:46
@wikingwhat's the gcc version?22:47
shogun-buildbotbuild #968 of cyg1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: Heiko Strathmann <>, Roman Votyakov <>22:47
shogun-buildbotbuild #1136 of bsd1 - libshogun is complete: Failure [failed test_1]  Build details are at  blamelist: Heiko Strathmann <>, Roman Votyakov <>22:54
-!- vgorbati [~vgorbati@] has quit [Ping timeout: 268 seconds]23:01
shogun-buildbotbuild #1259 of deb3 - modular_interfaces is complete: Success [build successful]  Build details are at
-!- sonney2k [] has quit [Read error: Operation timed out]23:07
shogun-notifier-shogun: Soeren Sonnenburg :develop * a2cf857 / src/Makefile.template,src/configure:
shogun-notifier-shogun: use full gcc command line for gcc -MM based dependency generation23:08
-!- sonney2k [] has joined #shogun23:10
shogun-notifier-shogun: Soeren Sonnenburg :develop * 8c54b9f / src/Makefile.template:
shogun-notifier-shogun: use CPP compflags in gcc -MM23:12
sonney2kwiking, now it is happy \o/23:20
shogun-buildbotbuild #969 of cyg1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: Soeren Sonnenburg <>23:20
shogun-buildbotbuild #970 of cyg1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: Soeren Sonnenburg <>23:22
sonney2kwiking, btw we should also enable tests to detect sse3/sse423:24
sonney2kthese can really make a difference23:24
sonney2kI don't understand though why -march=native doesn't do that23:27
shogun-buildbotbuild #1137 of bsd1 - libshogun is complete: Failure [failed test_1]  Build details are at  blamelist: Soeren Sonnenburg <>23:28
sonney2kactually it does!23:28
sonney2k...but it is written in the docs that this only works under linux23:29
shogun-buildbotbuild #1138 of bsd1 - libshogun is complete: Failure [failed test_1]  Build details are at  blamelist: Soeren Sonnenburg <>23:36
-!- votjakovr [] has left #shogun ["Fallen asleep!"]23:37
shogun-buildbotbuild #1260 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at  blamelist: Heiko Strathmann <>, Roman Votyakov <>23:44
shogun-notifier-shogun: Soeren Sonnenburg :develop * 1a9afee / src/configure:
shogun-notifier-shogun: compflags swig also needs -msse223:47
--- Log closed Tue Jun 25 00:00:03 2013