Open in new window / Try shogun cloud
--- Log opened Wed Mar 02 00:00:19 2016
-!- nelson-liu [~nelsonliu@c-73-53-48-140.hsd1.wa.comcast.net] has joined #shogun00:16
-!- Guest42037 [ibm___@nat/iiit/x-jvlytqzdmtlyqjyw] has quit [Ping timeout: 252 seconds]00:18
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Quit: Leaving.]00:21
-!- ebadawy [~ebadawy@197.163.32.61] has quit [Ping timeout: 276 seconds]00:42
-!- travis-ci [~travis-ci@ec2-54-161-110-71.compute-1.amazonaws.com] has joined #shogun00:54
travis-ciit's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/11295551300:54
-!- travis-ci [~travis-ci@ec2-54-161-110-71.compute-1.amazonaws.com] has left #shogun []00:54
-!- Rajat [73f87233@gateway/web/freenode/ip.115.248.114.51] has joined #shogun01:29
-!- Rajat [73f87233@gateway/web/freenode/ip.115.248.114.51] has quit [Client Quit]01:29
-!- nelson-liu [~nelsonliu@c-73-53-48-140.hsd1.wa.comcast.net] has quit [Remote host closed the connection]01:37
-!- lambday [6a33e982@gateway/web/freenode/ip.106.51.233.130] has quit [Ping timeout: 252 seconds]02:01
-!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 250 seconds]02:16
-!- shubham__ [uid57367@gateway/web/irccloud.com/x-erbfovrtgtdrllcz] has quit [Quit: Connection closed for inactivity]02:17
-!- travis-ci [~travis-ci@ec2-54-161-110-71.compute-1.amazonaws.com] has joined #shogun02:34
travis-ciit's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/11295768502:34
-!- travis-ci [~travis-ci@ec2-54-161-110-71.compute-1.amazonaws.com] has left #shogun []02:34
-!- travis-ci [~travis-ci@ec2-54-161-110-71.compute-1.amazonaws.com] has joined #shogun03:18
travis-ciit's Sergey Lisitsyn's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/11295868903:18
-!- travis-ci [~travis-ci@ec2-54-161-110-71.compute-1.amazonaws.com] has left #shogun []03:18
-!- shin_ [ca24b351@gateway/web/freenode/ip.202.36.179.81] has joined #shogun03:57
-!- shin_ [ca24b351@gateway/web/freenode/ip.202.36.179.81] has quit [Quit: Page closed]04:13
-!- nelson-liu [~nelsonliu@205.175.119.226] has joined #shogun04:13
-!- nelson-liu [~nelsonliu@205.175.119.226] has quit [Client Quit]04:14
-!- nelson-liu [~nelsonliu@205.175.119.226] has joined #shogun04:15
-!- nelson-liu [~nelsonliu@205.175.119.226] has left #shogun []04:15
-!- ebadawy [~ebadawy@197.163.32.61] has joined #shogun04:16
-!- amod95 [uid67143@gateway/web/irccloud.com/x-sjrqosjjqifwphxs] has joined #shogun04:40
-!- Izaya [~Eejya@103.225.100.51] has joined #shogun05:02
-!- Saurabh7 [~Saurabh7@103.225.100.51] has quit [Ping timeout: 276 seconds]06:07
-!- Rajat [73f87233@gateway/web/freenode/ip.115.248.114.51] has joined #shogun06:18
-!- Rajat [73f87233@gateway/web/freenode/ip.115.248.114.51] has quit [Client Quit]06:19
-!- Izaya [~Eejya@103.225.100.51] has quit [Ping timeout: 246 seconds]06:28
-!- viswanadh [~viswanadh@202.3.77.219] has joined #shogun06:39
-!- lzc [~lzc@174.139.209.123] has joined #shogun07:16
-!- bollu [~Adium@1.186.131.55] has quit [Ping timeout: 276 seconds]07:22
-!- lambday [c40f1067@gateway/web/freenode/ip.196.15.16.103] has joined #shogun07:29
-!- mode/#shogun [+o lambday] by ChanServ07:29
@wikinglambday: yo07:30
@lambdaywiking: yooo07:31
@lambdaysonney2k: yeah.. wasted a full year on earthly things :(07:31
@wikingwazza07:32
@lambdaywiking: going to join HeikoS this summer07:33
@wikingyeah heard it07:33
@lambdaywiking: for an internship in UCL07:33
@lambdaywiking: what's up with you? which part of the globe you're in presently?07:33
@wikingnow in europe again07:33
@wikingfor 3 months07:33
@lambdayman do you travel a lot :D07:34
@wikingmmm yeah07:34
@wikingbtw anybody knows how to get openining mode of a FILE*07:34
@wiking?07:34
@lambdaydon't know if that's possible07:38
-!- bollu [~Adium@1.186.133.185] has joined #shogun07:45
-!- amod95 [uid67143@gateway/web/irccloud.com/x-sjrqosjjqifwphxs] has quit [Quit: Connection closed for inactivity]07:48
-!- dev__ [developer@nat/iiit/x-woiynujiwrurjfkq] has joined #shogun07:49
-!- lzc [~lzc@174.139.209.123] has quit [Quit: Leaving.]07:51
-!- Saurabh7 [~Saurabh7@103.225.100.51] has joined #shogun07:51
-!- ebadawy [~ebadawy@197.163.32.61] has quit [Ping timeout: 248 seconds]07:56
Saurabh7lambday, around?07:57
-!- lzc [~lzc@115.27.195.193] has joined #shogun08:53
-!- dev__ [developer@nat/iiit/x-woiynujiwrurjfkq] has quit [Ping timeout: 246 seconds]08:58
sonne|work1wiking: fcntl with F_GETFL ?09:39
sonne|work1wiking: http://www.gnu.org/software/libc/manual/html_node/Getting-File-Status-Flags.html - so just the F_GETFL macro09:40
@wikingsonne|work1: this is for fd i presume09:50
@lambdaySaurabh7: hey09:50
sonne|work1wiking - sure09:51
sonne|work1wiking: but you can easily get the fd from FILE right?09:51
@wikingyep yep09:51
@wikingthe only problem is09:51
@wikingthat in shoung we are FILE* oriented09:51
@wikinginstead of fd09:51
@wikingie if we get a FILE* we dont automatically open fd09:52
@wikingbut if we get an fd we 'convert it' to FILE* :)09:52
sonne|work1wiking: it would make sense to change the code to just use one of either FILE or fd ...09:53
@wikingi mean we are using FILE* everywhere09:53
@wikingonly that we support to work on file that is given by fd09:53
Saurabh7lambday, quick question wrt linalg::dot10:12
Saurabh7lambday, i was trying to get dot product of vectors passed as ptr10:13
Saurabh7but there seems to be no matching function call although its implemented https://github.com/shogun-toolbox/shogun/blob/develop/src/shogun/mathematics/linalg/internal/implementation/Dot.h#L10610:13
Saurabh7do you need to expose it in redux module?10:14
-!- besser82 [~besser82@fedora/besser82] has joined #shogun10:32
-!- mode/#shogun [+o besser82] by ChanServ10:32
-!- viswanadh [~viswanadh@202.3.77.219] has quit [Ping timeout: 248 seconds]10:51
-!- lambday [c40f1067@gateway/web/freenode/ip.196.15.16.103] has quit [Ping timeout: 252 seconds]10:59
-!- Guest42037 [ibm___@nat/iiit/x-ccjufonpqsfjfvoj] has joined #shogun11:04
-!- lambday [c40f1067@gateway/web/freenode/ip.196.15.16.103] has joined #shogun11:18
-!- mode/#shogun [+o lambday] by ChanServ11:18
@lambdaySaurabh7: hey there11:18
@lambdaysorry I was in a meeting.. let me check your question11:18
@lambdaySaurabh7: yes you're right.. we don't have a wrapper for naked ptrs yet11:19
Saurabh7lambday, ok thats what i thought11:20
Saurabh7how much faster will it be than CMath :: dot ?11:20
@lambdaySaurabh7: I'll revisit these things in a few days.. maybe we can do some tricky stuffs so that one doesn't need to pass the length11:20
@lambdayusing std::begin() and std::end()11:20
@lambdaybut I guess this is c++14 stuffs11:20
@lambdaySaurabh7: speed won't be too much of an issue..11:21
@lambdaysince the backend is eigen, it will be just as fast if you create an eigen map and do eigen dot11:21
Saurabh7ok so the loops i see are jsut for the native11:21
Saurabh7oh alright11:22
@lambdaySaurabh7: we're thinking of getting rid of native btw..11:22
@wikingbtw where's the actual implementation of eigen dot11:23
@wiking?11:23
@lambdaywiking: you mean the place where we create the maps?11:24
@lambdaywiking: https://github.com/shogun-toolbox/shogun/blob/develop/src/shogun/mathematics/linalg/internal/implementation/Dot.h#L14111:24
@wikingno i mean in eigen itself11:24
@lambdayah that I'll have to check11:25
@lambdaymaybe in MatrixBase, not sure11:25
@lambdaywiking: https://bitbucket.org/eigen/eigen/src/30954d1a91e3d9af86eba2096396fdc50c6f8536/Eigen/src/Core/MatrixBase.h?at=default&fileviewer=file-view-default#MatrixBase.h-19811:28
Saurabh7lambday, also any reason why passing ptr should be better than SGVector itself ?11:28
@wikinglambday: implementation?11:28
@lambdaywiking: checking11:28
@lambdaySaurabh7: no reason.. but if in your impl you use, say, std::vector or std::unique_ptr<T[]> or naked ptr itself, you'd still be able to use linalg::dot with the same interface.. that was the goal11:29
@lambdaySaurabh7: it's far from the goal as I can see :|11:29
Saurabh7lambday, ok :)11:30
Saurabh7lambday, the reason i ask is when i replace smth like float64_t* avec=((CDenseFeatures<float64_t>*) lhs)->get_feature_vector(idx_a, alen, afree);11:31
Saurabh7with11:31
Saurabh7SGVector<float64_t> avec=((CDenseFeatures<float64_t>*) lhs)->get_feature_vector(idx_a);11:31
Saurabh7and use dot11:31
Saurabh7SGVector one is slower for some reason11:31
@lambdaywiking: https://bitbucket.org/eigen/eigen/src/30954d1a91e3d9af86eba2096396fdc50c6f8536/Eigen/src/Core/Dot.h?at=default&fileviewer=file-view-default11:32
@lambdaySaurabh7: it creates an eigen map and destroys it that causes some overhead maybe..11:32
@lambdayso you've noticed that native performs better?11:33
@lambdaythat's interesting!11:33
@lambdayis the difference significant?11:33
Saurabh7lambday, right now i am using ptrs and CMath::dot11:33
Saurabh7and its a bit faster11:33
Saurabh7ia m playing with kmeans and distance computations11:33
@lambdayyeah that just runs a loop..11:33
@lambdayokay11:34
@wikinglambday: i have couple of wondering thoughts in my head: a) would eigen be able to use sse4.2 dot product ? :)11:34
@wikingb) whether eigen expects the underlying memory aligned?11:34
@lambdaywiking: I had a hunch that you'd be asking about simd thing :D11:35
@lambdaynot sure.. I'll have to check.. maybe lisitsyn knows..11:36
@lambdaySaurabh7: I'd bet that the overhead is due to temporary object creation.. gcc vectorizes the loop in a similar way for native also I guess.. so if eigen guys are not doing anything explicit, the product code would be same11:38
Saurabh7lambday, i will comment in a issue with the differences because it can be  a bit significant11:39
Saurabh7lambday, oh alright11:39
@lambdaySaurabh7: that would be great! It might give some motivation to keep the poor NATIVE backend still alive..11:39
@lambdaySaurabh7: btw have you tried viennacl dot? you may be surprised11:40
@lambdaydot<Backend::VIENNACL>(a,b)11:41
Saurabh7lambday, so you wouldn't expect better with eigen and naked ptr ?11:41
Saurabh7no i havent , ned to try11:41
@lambdaySaurabh7: I never thought that the map creation part can pose any significant overhead.. so I expected eigen dot to rather be a bit better than native, for large enough vectors..11:42
@lambdaybut small ones, it may be a bit slow11:42
@lambdaySaurabh7: check with vectors of size 1bil11:43
Saurabh7lambday, ok11:45
Saurabh7but i wasnt testing the dot in isolation11:45
Saurabh7regarding the issue https://github.com/shogun-toolbox/shogun/issues/298711:45
Saurabh7so there might be other things going on11:45
@lambdaySaurabh7: great discussion man!11:46
@lambdayfor these things, we need profiler11:47
Saurabh7lambday, :) i was using the dot in distance computations for dataset arnd 2000 isntances11:48
Saurabh720k11:48
Saurabh7lambday, ye i saw your benchamrks11:48
Saurabh7i  need to try that out11:48
@lambdayif we can figure out the part of the algorithm that stalls us for the longest, we'll focus on that.. eigen dot/native dot may not be the bottleneck11:49
Saurabh7yup we figured most of the parts and after changing the distance things too its much better11:52
Saurabh7lambday, but the dot thing came up because i used linalg::dot with sgvectors and the performace decreased11:54
Saurabh7but  its  not a big issue11:54
Saurabh7thanks though :)11:55
-!- Izaya [~Eejya@103.225.100.51] has joined #shogun11:55
sonne|work1performance of dot largely depends on #dims of the data...12:02
sonne|work1I mean with 2d data the overhead of even using eigen is probably making dot look lousy12:02
-!- viswanadh [~viswanadh@202.3.77.206] has joined #shogun12:05
-!- Izaya [~Eejya@103.225.100.51] has quit [Ping timeout: 248 seconds]12:10
Saurabh7I used data with ~70 attributes tho12:14
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun12:20
-!- mode/#shogun [+o HeikoS] by ChanServ12:20
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Client Quit]12:20
-!- Guest42037 [ibm___@nat/iiit/x-ccjufonpqsfjfvoj] has quit [Ping timeout: 260 seconds]12:29
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has joined #shogun12:43
-!- mode/#shogun [+o HeikoS] by ChanServ12:43
-!- ibm [ibm___@nat/iiit/x-yacpzsqlzsenqwhw] has joined #shogun13:14
-!- ibm is now known as Guest3561013:15
-!- Mannu_ [bce2dc1b@gateway/web/freenode/ip.188.226.220.27] has joined #shogun13:19
-!- Guest35610 [ibm___@nat/iiit/x-yacpzsqlzsenqwhw] has quit [Ping timeout: 240 seconds]13:25
-!- Mannu_ [bce2dc1b@gateway/web/freenode/ip.188.226.220.27] has quit [Ping timeout: 252 seconds]13:28
-!- lambday [c40f1067@gateway/web/freenode/ip.196.15.16.103] has quit []13:41
-!- HeikoS [~heiko@host-92-0-162-192.as43234.net] has quit [Quit: Leaving.]13:45
shogun-buildbotbuild #1122 of precise - libshogun is complete: Success [build successful]  Build details are at http://buildbot.shogun-toolbox.org/builders/precise%20-%20libshogun/builds/112213:52
-!- lzc1 [~lzc@98.126.68.18] has joined #shogun13:55
-!- lzc [~lzc@115.27.195.193] has quit [Ping timeout: 248 seconds]13:56
-!- bollu [~Adium@1.186.133.185] has quit [Quit: Leaving.]13:57
shogun-buildbotbuild #1144 of FCRH - libshogun is complete: Failure [failed test]  Build details are at http://buildbot.shogun-toolbox.org/builders/FCRH%20-%20libshogun/builds/1144  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>, youssef <youssef.emad.attia@gmail.com>14:07
-!- lzc1 [~lzc@98.126.68.18] has quit [Ping timeout: 248 seconds]14:09
sonne|work1Saurabh7: maybe we should benchmark starting with 2 dims going to say 100k dims logarithmically14:14
-!- Nebula_ [~chatzilla@1.39.60.247] has joined #shogun14:19
-!- witness_ [uid10044@gateway/web/irccloud.com/x-zaovraucsiyssjaz] has joined #shogun14:21
-!- lzc [~lzc@115.27.195.193] has joined #shogun14:24
-!- Nebula_ [~chatzilla@1.39.60.247] has quit [Ping timeout: 248 seconds]14:28
-!- Guest35610 [ibm___@nat/iiit/x-dyvljvkssbnfrfpg] has joined #shogun14:35
-!- bollu [~Adium@1.186.5.190] has joined #shogun14:50
-!- travis-ci [~travis-ci@ec2-54-196-97-16.compute-1.amazonaws.com] has joined #shogun14:50
travis-ciit's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/11310823314:50
-!- travis-ci [~travis-ci@ec2-54-196-97-16.compute-1.amazonaws.com] has left #shogun []14:50
-!- bollu [~Adium@1.186.5.190] has quit [Ping timeout: 244 seconds]14:54
-!- viswanadh [~viswanadh@202.3.77.206] has quit [Ping timeout: 276 seconds]14:59
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has joined #shogun15:46
-!- mode/#shogun [+o HeikoS] by ChanServ15:46
-!- ebadawy [~ebadawy@197.163.32.61] has joined #shogun16:13
@wikingHeikoS: ping16:13
-!- witness_ [uid10044@gateway/web/irccloud.com/x-zaovraucsiyssjaz] has quit [Quit: Connection closed for inactivity]16:28
Saurabh7sonne|work1, yes that would be a good idea , will add it to my list :)16:28
-!- Nebula_ [~chatzilla@1.39.62.101] has joined #shogun16:30
-!- Nebula_ [~chatzilla@1.39.62.101] has quit [Client Quit]16:33
-!- viswanadh [~viswanadh@202.3.77.219] has joined #shogun16:39
@HeikoSwiking: ping16:41
@HeikoSwiking: whazup?16:41
@wikingmmm16:45
@wikingwhat did i want? )16:45
@wikinggood question16:45
@HeikoSwiking: haha ;)16:47
@HeikoSwiking: I have few things: eigen3, html upload, and c++1116:47
@HeikoSwiking: oh and did you see this docker cloud thing?16:48
lisitsynmeh16:50
@HeikoSlisitsyn: muuuuuh16:51
@HeikoSwiking: notebooks on website are broken, do you have any idea on that?16:52
@HeikoSbuild seems to work16:52
-!- ebadawy [~ebadawy@197.163.32.61] has quit [Ping timeout: 276 seconds]16:59
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has quit [Ping timeout: 240 seconds]17:00
-!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 250 seconds]17:16
-!- viswanadh [~viswanadh@202.3.77.219] has quit [Ping timeout: 260 seconds]17:21
-!- Guest35610 is now known as ibm_17:28
-!- dungeon_master [developer@nat/iiit/x-elibthlofnjafmam] has joined #shogun17:30
-!- bollu [~Adium@1.186.2.212] has joined #shogun17:33
-!- Tanishq [~tanishq@1.186.14.246] has joined #shogun17:38
-!- Saurabh7 [~Saurabh7@103.225.100.51] has quit [Quit: Leaving]17:44
-!- Izaya [~Eejya@103.225.100.51] has joined #shogun17:48
-!- lzc [~lzc@115.27.195.193] has quit [Quit: Leaving.]17:51
-!- bollu [~Adium@1.186.2.212] has quit [Quit: Leaving.]17:53
-!- lzc [~lzc@115.27.195.193] has joined #shogun17:54
-!- Tanishq [~tanishq@1.186.14.246] has quit [Ping timeout: 250 seconds]17:55
-!- lzc [~lzc@115.27.195.193] has quit [Client Quit]17:55
-!- Izaya [~Eejya@103.225.100.51] has quit [Ping timeout: 276 seconds]17:58
-!- Tanishq [~tanishq@1.186.12.127] has joined #shogun17:59
-!- viswanadh [~viswanadh@202.3.77.206] has joined #shogun18:02
-!- Izaya [~Eejya@103.225.100.51] has joined #shogun18:12
-!- lambday [6a33e9a7@gateway/web/freenode/ip.106.51.233.167] has joined #shogun18:39
-!- mode/#shogun [+o lambday] by ChanServ18:39
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has joined #shogun18:40
-!- mode/#shogun [+o HeikoS] by ChanServ18:40
-!- Tanishq [~tanishq@1.186.12.127] has quit [Ping timeout: 268 seconds]19:32
-!- dungeon_master [developer@nat/iiit/x-elibthlofnjafmam] has quit [Ping timeout: 268 seconds]19:37
-!- Tanishq [~tanishq@1.186.14.33] has joined #shogun19:38
-!- dungeon_master [developer@nat/iiit/x-hdzsrkizmqrnydeo] has joined #shogun19:41
-!- cache-nez [~cache-nez@93.175.2.172] has joined #shogun19:45
-!- HeikoS [~heiko@untrust-out.swc.ucl.ac.uk] has quit [Ping timeout: 244 seconds]19:53
-!- bollu [~Adium@1.186.130.50] has joined #shogun20:05
-!- Izaya [~Eejya@103.225.100.51] has quit [Ping timeout: 252 seconds]20:06
-!- Izaya [~Eejya@103.225.100.51] has joined #shogun20:18
-!- besser82 [~besser82@fedora/besser82] has joined #shogun20:24
-!- mode/#shogun [+o besser82] by ChanServ20:24
-!- travis-ci [~travis-ci@ec2-54-205-222-49.compute-1.amazonaws.com] has joined #shogun20:40
travis-ciit's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: https://travis-ci.org/shogun-toolbox/shogun/builds/11315541420:40
-!- travis-ci [~travis-ci@ec2-54-205-222-49.compute-1.amazonaws.com] has left #shogun []20:40
bolluheh, that's a cool mesage20:54
-!- besser82 [~besser82@fedora/besser82] has quit [Ping timeout: 246 seconds]21:01
-!- viswanadh [~viswanadh@202.3.77.206] has quit [Ping timeout: 244 seconds]21:01
-!- viswanadh [~viswanadh@202.3.77.206] has joined #shogun21:14
-!- Mannu_ [0e8bf032@gateway/web/freenode/ip.14.139.240.50] has joined #shogun21:15
-!- viswanadh [~viswanadh@202.3.77.206] has quit [Ping timeout: 276 seconds]21:27
-!- Mannu_ [0e8bf032@gateway/web/freenode/ip.14.139.240.50] has quit [Quit: Page closed]21:36
-!- Shikher [0e8bf032@gateway/web/freenode/ip.14.139.240.50] has joined #shogun21:38
-!- Shikher [0e8bf032@gateway/web/freenode/ip.14.139.240.50] has quit [Quit: Page closed]21:53
-!- ibm_ [ibm___@nat/iiit/x-dyvljvkssbnfrfpg] has quit [Ping timeout: 260 seconds]21:55
shogun-buildbotbuild #589 of debian wheezy - memcheck is complete: Failure [failed memory check]  Build details are at http://buildbot.shogun-toolbox.org/builders/debian%20wheezy%20-%20memcheck/builds/589  blamelist: youssef <youssef.emad.attia@gmail.com>22:05
@wikinglisitsyn: ping22:22
-!- lambday [6a33e9a7@gateway/web/freenode/ip.106.51.233.167] has quit [Ping timeout: 252 seconds]22:28
-!- bollu [~Adium@1.186.130.50] has quit [Quit: Leaving.]23:44
-!- Izaya [~Eejya@103.225.100.51] has quit [Quit: Leaving]23:54
--- Log closed Thu Mar 03 00:00:21 2016