Open in new window / Try shogun cloud
--- Log opened Tue Sep 06 00:00:37 2016
-!- yorkerlin [ae73d4c2@gateway/web/freenode/ip.] has joined #shogun04:03
yorkerlinping wiking04:04
@wikingpong what's up04:04
yorkerlinabout the optimization framework04:04
yorkerlinr you working on kkt solvers?04:05
@wikingheheh wish04:05
@wikingtill now it was pushing bits areound04:05
@wikingfor gsoc04:05
@wikingand some javascript related stuff with shogun + windows port of shogun04:06
@wikingso not yet04:06
yorkerlinic. so we can discuss about the framework04:06
@wikingso the thing is04:07
@wikingthat since now we have a full refact04:07
@wikingof the linalg backend04:07
@wikingwe should rely on that04:07
@wikingso for any matrix factorization04:07
@wikingor basically any linalg operations we should use the new linalg backend04:08
@wikingin the beginning it's going to be a pain a bit04:08
@wikingbecause it'll require us to implement some of the linalg methods of our own04:08
@wikingbut it's worth it in the long run04:08
@wikingbecause then one will be able to choose between the backend being used for linalg calculations (like cpu, gpu etc.)04:08
yorkerlindo we assume that we can use the linalg lib in the opt framework04:09
@wikingas well as openmp04:09
@wikingand c++1104:09
yorkerlinmaybe we should list most of operators used in the opt framework04:10
@wikingfor kkt there are usually LDL, QR factorization04:10
@wikingso basically matrix factorization04:10
yorkerlinI may need LDLT04:11
yorkerlinHeiko told me that now tensorflow supports API for building computational graphs in C++04:12
@wikingwe could rely on that04:14
yorkerlinso we should have some backends for gradient computation (eg, tensorflow and hand gradient )04:14
@wikingso in my mind04:15
@wikingif some of the heavy computation stuff we cannot only cover with linalg lib04:15
@wikingwe should wrap it somehow04:16
@wikingso we could later easily factor it out04:16
@wikingand have a different implementation of it04:16
@wikingsince now most of the case is the problem is that all the computation is hard-coded04:16
@wikinghence it's very hard to factor it out04:16
@wikingok not very hard04:16
@wikingbut cumbersome04:16
@wikingand nowadays if i look at libsvm, svmlight implementation04:16
@wikingmy heart starts to bleed :)04:17
yorkerlinso add a tensorflow backend in the linalg lib?04:17
@wikingmmm i'm not so sure if we wanna add it there04:17
@wikingbut i'll check how we can integrate it04:18
@wikingmy only question is04:18
@wikingfor example04:18
@wikingin case of TF04:18
@wikingwould it be possible to distribute the computations?04:19
@wikingi mean it says "data flow graphs for scalable machine learning"04:19
@wikingbut to be honest if one asks me i much more prefer an actor based framework for distributing the computations between nodes04:19
yorkerlinmultiple GPUs?04:19
@wikinggpus, cpus whatever :)04:20
yorkerlinthe focus of Shogun is single machine, right?04:21
@wikingmmm should not be :)04:21
@wikingi mean yes currently we only support that04:22
@wikingbut would be good to start not only support that04:22
@wikinghence why i'm telling here that all the computations04:22
@wikingshould be wrapped04:22
@wikinginto some abstraction04:22
@wikingso that later having different way of doing the computations04:22
@wikingwould be easy to change04:22
yorkerlindo we pay attention to PC clusters instead of HPC?04:24
@wikingbut we should not focus now on04:25
@wikinghow the clustering is being done04:25
@wikingjust make it possible04:25
@wikingfrom the code point of view04:25
yorkerlinwe should do a lot of refactoring.04:27
@wikingyep :)04:27
yorkerlinI will be back in 10 mins.04:28
@wikingthe best is to do what we did with the linalg refactor gsoc project04:29
yorkerlindoes the linalg refactor project listed in the wiki?04:36
yorkerlin ?04:37
@wikingit was this one04:38
@wikingoxphos wrote a whole wiki page04:39
@wikingabout the project04:39
@wikingyou can find it here... lemme try to find t04:39
yorkerlinwe should  investigate how to integrate tensoflow in the meantime04:40
@wikingthis is the report of that project04:41
yorkerlinI will take a look at the report04:42
yorkerlinwe may need a tensorflow backend if gradient information is needed.04:43
yorkerlinBTW, do you know this project,
yorkerlinGPflow uses the python API from tensorflow. BTW, matrix solvers are needed in GP.04:47
yorkerlinI mean that tensowflow should support many matrix solvers04:48
yorkerlinat least in the python API04:49
yorkerlinOnce the C++ API is polished, we could use the C++ API to build a tensorflow backend.04:50
@wikingbut what's tensorflow giving04:57
@wikingin extra here?04:57
yorkerlinmaybe we should have a gradient computation framework?04:58
yorkerlinfor hand gradient, the linalg is used. for AD, tensorflow is used04:59
yorkerlinautomatic differentiation04:59
@wikingok i need to dig in05:00
@wikinghow we could use tf's computation graph05:00
@wikingvia the c++ api05:00
@wikingbecause that support just recently has been released05:00
@wikingtill now it was only via python api05:00
yorkerlinno. accodring to the release note05:01
yorkerlinAdded graph-construction C API Major revision to the graph-construction C++ API05:02
yorkerlinI am looking for examples about how to use the C++/C API05:02
@wikingi mean it has been released05:03
@wikingAdded graph-construction C API05:03
@wikingthis wasnt part till now05:04
yorkerlinI mean tensorflow now has C++/C API for graph construction05:04
@wikingthat's what i mean that till now it wasn't there05:04
@wikingso i have 0 idea05:04
@wikinghow the api is like05:04
yorkerlinyes. it is true. that is why I am looking for examples about the usage of the API05:05
yorkerlinThere are two ways to add a tensorflow backend.05:07
yorkerlinone way is that each algorithm has at least two implemenations. one for hand gradient, the other for tensorflow.05:08
yorkerlinThere are two ways to use tensorflow in shogun. one way is that each algorithm has at least two implemenations. one for hand gradient, the other for tensorflow. another way is to add a tensorflow backend in the linalg lib.05:09
yorkerlindo you think which way is better? or other suggestion?05:11
yorkerlinor an AD framework (eg, tensorflow backend, CNTK backend, and Stan backend)05:17
yorkerlinLet me say it again.05:24
yorkerlin1 a tensorflow backend for the linalg; 2 an implementation for algorithms in shogun using tensorflow; 3 an AD framework which can use tensorflow, or CNTK, or STAN?05:25
yorkerlinI have to leave now. I will be in ICR again soon.05:26
-!- yorkerlin [ae73d4c2@gateway/web/freenode/ip.] has quit [Quit: Page closed]05:26
@wikinglisitsyn, ping12:53
-!- HeikoS [] has joined #shogun13:16
-!- mode/#shogun [+o HeikoS] by ChanServ13:16
@wikingHeikoS: pong ping ping pong13:37
-!- HeikoS [] has quit [Ping timeout: 265 seconds]13:45
-!- sanuj [~sanuj@] has joined #shogun14:02
-!- jones_ [5befb10e@gateway/web/freenode/ip.] has joined #shogun14:44
-!- jones_ [5befb10e@gateway/web/freenode/ip.] has quit [Ping timeout: 264 seconds]14:51
-!- jones_ [5befb10e@gateway/web/freenode/ip.] has joined #shogun14:52
jones_Hi, is there anybody here who could help me with loading a trained CMulticlassLibSVM after saving it to a file? Saving seems to work fine, but I'm not sure how you are supposed to load it. I couldn't find an example on the web or in the docu. Here is my current approach
@wikingand i guess model is an std::string that contains the path for the model file?15:07
jones_Oh sry, yes its just a path15:08
@wikingok and what's the error?15:09
jones_I get the following error: [ERROR] In file .../shogun/src/shogun/kernel/Kernel.h line 211: LinearKernel::kernel(): index out of Range: idx_a=240/240 idx_b=0/24015:10
@wikingyou get this while loading?15:10
jones_No its occuring in apply_multiclass15:10
-!- HeikoS [] has joined #shogun15:11
-!- mode/#shogun [+o HeikoS] by ChanServ15:11
-!- HeikoS [] has quit [Client Quit]15:12
jones_But I'm not sure if I'm setting the kernel and responses correctly, how are you supposed to set those while predicting?15:12
@wikingbut what are you actually applying?15:24
@wikingthe kernel that was in the trained machine?15:24
@wikingwill overwrite your CMulticlassLibSVM15:24
@wikingso you should set the kernel15:24
jones_I initialize the kernel before with the test data: kernel->init(testfeatures,testfeatures);15:25
@wikingyeah but you set the kernel of the CMulticlassLibSVM15:26
@wikingthe loading of the pretrained data15:26
jones_    shogunsvm->load(f);          kernel->init(testfeatures,testfeatures);         shogunsvm->set_kernel(kernel);         CMulticlassLabels*  labels=shogunsvm->apply_multiclass();15:31
jones_has the same effect, hm15:31
jones_maybe the saved model isn't even right, the support vector indices are kinda high15:32
@wikingdunno when and how you have created the svm15:37
jones_I did an apply_multiclass with the trained svm and I get repsonses, so it should work15:40
@wikingyou could btw do a shogunsvm->apply_multiclass(testfeatures)15:41
jones_yes, I also tried a single apply_one(0). Same error15:42
@wikingapply_one(0) ?15:43
@wikingit will give you the same error15:43
@wikingif you give a null input15:43
jones_how am I supposed to input the test data? kernel->init?15:44
@wikingjust have the testfeatures15:45
jones_so shogunsvm->apply_multiclass(testfeatures); ?15:45
@wikingvirtual CMulticlassLabels* apply_multiclass(CFeatures* data=NULL);15:45
@wikingand yes15:45
jones_and how do I set the kernel then?15:45
jones_If I don't initialize the kernel I will get errors about its left and right hand sizes15:47
jones_[ERROR] In file .../shogun/src/shogun/multiclass/MulticlassSVM.cpp line 86: MulticlassLibSVM: No left hand side specified15:49
@wikingkernel should have been serialized15:55
@wikingas part of the model15:55
jones_I get: "shogun/src/shogun/multiclass/MulticlassSVM.cpp line 82: No kernel assigned! " if i don't set an explicit  kernel and as far I can follow it, CMulticlassSVM::load doesn't seem to set the kernel16:02
-!- sanuj [~sanuj@] has quit [Ping timeout: 255 seconds]16:08
@wikingmmm that's actually a problem :(16:15
@wikingjones_: only good news i have now for you16:23
@wikingthat soon we'll gonna have a new release :)16:23
@wikingand that has new serialization backend :)16:23
jones_hooray, thanks for your help16:25
@wikingnot much i could helped it seems16:26
jones_you answered my questions, its help enough ;) and you could report the bug. anyway I must leave now, thanks again and bye16:28
@wikingjones_: could you fill a bug16:30
@wikingin issues plz16:30
@wikingwould be really appreciated16:30
@wikingand you would have an update16:30
@wikingas soon as it's fixed16:30
jones_I will look into it16:31
-!- jones_ [5befb10e@gateway/web/freenode/ip.] has quit [Quit: Page closed]16:31
-!- sanuj [~sanuj@] has joined #shogun16:46
-!- HeikoS [] has joined #shogun18:35
-!- mode/#shogun [+o HeikoS] by ChanServ18:35
-!- sanuj [~sanuj@] has quit [Ping timeout: 240 seconds]18:37
-!- HeikoS [] has quit [Quit: Leaving.]18:49
-!- sanuj [~sanuj@] has joined #shogun18:50
-!- HeikoS [] has joined #shogun19:01
-!- mode/#shogun [+o HeikoS] by ChanServ19:01
-!- HeikoS [] has quit [Quit: Leaving.]20:48
-!- HeikoS [] has joined #shogun20:52
-!- mode/#shogun [+o HeikoS] by ChanServ20:52
-!- HeikoS [] has quit [Quit: Leaving.]20:57
-!- HeikoS [] has joined #shogun21:12
-!- mode/#shogun [+o HeikoS] by ChanServ21:12
-!- sanuj [~sanuj@] has quit [Remote host closed the connection]21:12
-!- HeikoS [] has quit [Client Quit]21:14
--- Log closed Wed Sep 07 00:00:38 2016