Open in new window / Try shogun cloud
--- Log opened Tue May 03 00:00:42 2011
-!- alesis-novik [~alesis@188.74.87.84] has quit [Remote host closed the connection]00:34
-!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has joined #shogun01:43
-!- alesis-novik [~alesis@188.74.87.84] has joined #shogun03:44
-!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has quit [Ping timeout: 276 seconds]03:51
-!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has joined #shogun04:25
-!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has quit [Ping timeout: 260 seconds]09:44
-!- blackburn [~qdrgsm@188.168.5.65] has joined #shogun09:46
-!- mikiobraun [~mikio@squid.ml.tu-berlin.de] has joined #shogun10:28
-!- blackburn [~qdrgsm@188.168.5.65] has left #shogun []10:38
-!- mikiobraun [~mikio@squid.ml.tu-berlin.de] has quit [Quit: Leaving.]12:00
alesis-novikGood day14:56
-!- vojta [~quassel@2001:718:2:1634:221:6aff:fe58:151a] has joined #shogun15:05
-!- vojta is now known as VojtechFranc15:05
VojtechFranchi alesis15:06
alesis-novikHello VojtechFranc15:06
VojtechFrancI prepared Matlab implementation of the GMM-EM algorithm plus some examples.15:07
alesis-novikcan you send it over email? I'll have a look at it15:08
VojtechFrancIt is a small Matlab toolbox. you can install it from SVN: svn co http://ptak.felk.cvut.cz/datagrid/svn/stprtool/15:09
VojtechFrancusername is stprtool, no pasword15:09
VojtechFrancthe algorithms relevant to your work are in the folder "probab"15:09
VojtechFrancthere are functions for deadling with the GMM and the EM algorithm itself15:10
VojtechFrancthere are also few examples of usage on systhetical 1d and 2d data15:10
VojtechFrancin addition there is one example using the GMM-EM algorithm to train OCR classifier on USPS database15:11
alesis-novikI'll have a look at it then.15:11
VojtechFrancGreat! Do not hasiate to ask questions.15:12
VojtechFrancCurently there is a basic implementation of the EM which you shoul re-implement to C++.15:13
VojtechFrancThere a few differneces compared to textbook implementaion of GMM-EM.15:13
VojtechFranc1) the covariance matrix is represented in decomposed form which is uself fo several reasons (speed, numerical stability, ...)15:14
VojtechFranc2) Most of the computations are done in log domain which is more robust15:15
VojtechFranc3) there is a kind of regularization (namely, bound on the minimal variance) which makes the log-likelihood problem well posed15:15
VojtechFrancIn the time before you start implementing, you could try to get familiar with the code.15:17
alesis-novikso log domain deals with high/low numbers and regularization deals with singular Gaussians then?15:18
VojtechFrancDo you need to recomend some literature on EM ? I think it would be necessary only if you are not familiar with the EM15:18
VojtechFrancyes, exactly15:19
VojtechFrancif we have time, I'd like you to implement other imporvements: like merge/split rules etc.15:20
alesis-novikI've studied EM from C. M. Bishops "Pattern Recognition and Machine Learning", so I think I know the general theory behind it15:20
VojtechFrancyes, this should be enough15:21
VojtechFrancyou can ask me if you find something strange in the code15:22
VojtechFrancbut it should not be difficult15:22
alesis-novikmerge/split would be dynamically changing number of components?15:22
VojtechFrancexactly15:22
VojtechFrancit is useful to avoid some "bad" local maxima of the log-likelihood15:23
VojtechFrancand to remove gaussian components not supported by enough data15:23
alesis-novikSo it would be a more robust way of dealing with singular Gaussians than minimal variance as well15:24
VojtechFrancNo, it does not improve robustness.15:25
VojtechFrancIt makes the model more meaningfull (by removing the singular components) and helps to escape some local maxima15:25
alesis-novikI think I should have time to implement that15:27
VojtechFrancit would be nice. However, let us proceed gradually: i) implementing the basic version, ii) testing/ debuggin and iii) implementing other improvements15:29
alesis-novikwell, I guess the *very* basic version is already there15:29
VojtechFrancby basic I mean the one I sent you15:30
VojtechFrancit would be also nice if you implement some demosntation scripts15:31
VojtechFrancboth toy examples and close to some real application15:31
VojtechFrancthis will require some graphical output15:32
VojtechFrancdo you prefer Matlab to Python or something else?15:32
alesis-novikI think I would prefer Matlab, though I can't really use it from home. I'm not really that good at Python though.15:33
VojtechFrancgreat, I also prefer Matlab15:34
alesis-novikLike I said, the only problem is that I can't really do Matlab because I can only use it in the University15:35
VojtechFrancThen decide what is better for you. I can use Python as well.15:36
alesis-novikI assume I can do the demonstration scripts at the end, after doing the implementation?15:38
VojtechFrancYes, but it is useful to have at least some simple example which you can used during development15:39
VojtechFranclike some synthetic data where you can visualized the found model to see whether the estimate makes a sense ...15:40
VojtechFrancthe code I sent you contains such examples15:40
alesis-novikI wonder if R could be used for this?15:41
VojtechFrancno idea, I know nothing about R.15:41
VojtechFrancAs I said I would prefer i) Matlab or ii) Python or iii) Octave15:42
VojtechFrancSo, that's it from my side. I'll have to disconnect soon.15:43
VojtechFrancPlease, go though the code and don't hasitate to send me an email if you meet any problem.15:44
alesis-novikHmm, Octave looks quite similar to Matlab, and given that it's not going to be used for computation, just visualization, it might port to Matlab easily15:44
alesis-novikSo the implementation should also have an input parameter where you can specify if the cov is "full" "diag" or "spherical"?15:45
VojtechFrancI thing the same holds for Matlab and python - they will be used just for visualization15:45
VojtechFrancthe code will be in C++15:45
VojtechFrancin the shogun library linked to the corresponding environment15:45
alesis-novikBut I can't really use Matlab from where I'll be developing this and I'm not too comfortable with Python. I think I'll deal with this issue when time comes15:46
VojtechFrancok, think about other alternatives15:47
alesis-novikso about the covariance types, I should use the 3 used in the Matlab code, correct?15:48
VojtechFrancfor now yes, in the future I'd like to ask other options lake the Factor Analysis Decomposition etc15:48
VojtechFrancit sould not be problem to add other representations if the code is well implemented15:49
alesis-novikI'll think about separating different parts so they can be expanded on later15:50
VojtechFrancgreat, the code I sent you might give you an inspiration beause it allows simple extensions15:51
VojtechFrancunfortunatelly, I need to go now. Shell we agree on another chat (besides emails) ?15:52
alesis-novikI think we can agree on it over the emails. I'll probably only have time to analyse the Matlab code during the next 2 weeks because of my exams15:53
VojtechFrancyes, of course. Let's communicate via emails.15:54
alesis-novikWhen the actual coding starts we could have weekly talks or so15:54
VojtechFrancyes, sure.15:54
VojtechFrancSo, I wish you succesful examination period.15:55
alesis-novikThanks. 4 exams to go.15:55
VojtechFrancthen good luck!15:56
VojtechFrancbye15:56
@bettybooVojtechFranc, back to real-life?15:56
alesis-novikSee you15:56
VojtechFrancbettyboo, sorry don't know who is hiding behind this nice nickname15:57
@bettybooVojtechFranc: most however try to improve kpca in terms of robustness and not run-time performance15:57
VojtechFrancbettyboo, I concentrate on the EM becasue KPCA doesn't seem to be very practical algorithm15:59
@bettybooVojtechFranc: Going to discuss details of EM with Vojtech tomorrow15:59
alesis-novikA bot that is now saying scary relevant things15:59
VojtechFrancbattyboo, sorry have to disconnect now. See you16:00
-!- VojtechFranc [~quassel@2001:718:2:1634:221:6aff:fe58:151a] has quit [Remote host closed the connection]16:00
alesis-novikbettyboo, you scare me sometimes16:00
@bettybooalesis-novik: now I am already scared of having to wade through all of them16:00
-!- alesis-novik [~alesis@188.74.87.84] has quit [Quit: I'll be Bach]16:02
-!- blackburn [~qdrgsm@188.168.2.173] has joined #shogun17:44
-!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has joined #shogun18:12
-!- ameerkat [~ameerkat@184-98-140-155.phnx.qwest.net] has quit [Ping timeout: 260 seconds]18:33
-!- blackburn [~qdrgsm@188.168.2.173] has left #shogun []20:33
-!- blackburn [~qdrgsm@188.168.2.173] has joined #shogun21:49
@sonney2kSo Blackburn you are now the one to take over my job22:57
blackburnsonney2k: oh..22:57
blackburnsonney2k: your baby is born?22:57
@sonney2kYes exactly22:58
blackburnsonney2k: congrats!22:58
@sonney2kSleeping next to me currently.22:58
@sonney2kThanks.22:59
blackburnsonney2k: is it a boy or girl?22:59
@sonney2kBoy23:00
blackburnnice, did you chose a name?23:00
@sonney2kYes Jan23:00
blackburnnice! congrats to you again and pass congrats to your wife from cold russia :)23:01
@bettyboo:Q blackburn23:01
@sonney2kI am still full of adrenaline so cannot sleep.23:01
blackburnsonney2k: I'll try to take some your work23:02
@sonney2kThanks ... I hope we can leave the hospital this week then I also have my computer plus kbd back23:05
@sonney2kAnyways good night23:06
blackburnsonney2k: no problem, I will just react on some things there will be23:07
blackburnsonney2k: good night! let you have some sleep23:07
--- Log closed Wed May 04 00:00:43 2011