Artificial Intelligence

Pexbo said:
My AI lecturer thinks the fear of the singularity moment is stupid and Musk and Hawking use it to create a mystique and promote interest around an important field.
Click to expand...
Nah, I think that eventually we will arrive there, although who knows when.

Hawking has always been a doom-monger to be hair. He isn't Nick Bostrom in that aspect, but still (he has similar view for aliens as for AI, that they will kill us).

Not sure about Musk, he funded a company (pledging 1B dollars in the process), recruiting some of the top young ML scientists in the world (Sutskever, Goodfellow, Karpathy etc).
 
Revan said:
@dustfingers

Just came back at home from a lecture given by Michael Jordan. That man is a fecking machine. I had the impression from that lecture that pretty much no one in the room (a few dozens of them with PhD, many professors at universities) understand most of the stuff he was doing and the way he was easily to make relations between stuff from statistics, function analysis, optimizations to each other but also to other disciplines like quantum mechanics was fascinating. In addition, he seems to speak several languages and being a great communicator.

The biggest show of intelligence I have ever seen in my life.
Click to expand...

adexkola said:
Where were you?? Lucky bastard.
Click to expand...

Just saw that the lecture has been available online. It is still the most impressive lecture I have ever followed and definitely worth 90 minutes.

 
@Revan I finished the lecture this weekend.

A favorite book of mine is "Advice to Rocket Scientists" (the author's advice literally got me a job). The author states that during a successful defense of a dissertation, in order to demonstrate your mastery of a certain subject, at some point you have to "lose the audience", including the professors reviewing your work. The general theme of your research should be grasped by anyone with a fundamental understanding of the subject, but your methodology and or results should tread on new ground. That is the impression I got watching the lecture. 2 things in particular blew my mind:

- Derivation of statistical bounds using optimization (min-max) and stability theory
- Using variational analysis of continuous functions to obtain more general forms of limited heuristic solutions

At some points I felt like I needed to download the papers/references noted by Professor Jordan and read them to fully understand his methodology, but I never felt like he didn't fully understand the underlying concept. With some modification he could give this same lecture to a complete novice, and the novice would grasp what Professor Jordan was trying to do: proof of true mastery according to Feynman.

I also felt (and still feel) like an utter dummy after watching the lecture. How does one get to that point?
 
adexkola said:
@Revan I finished the lecture this weekend.

A favorite book of mine is "Advice to Rocket Scientists" (the author's advice literally got me a job). The author states that during a successful defense of a dissertation, in order to demonstrate your mastery of a certain subject, at some point you have to "lose the audience", including the professors reviewing your work. The general theme of your research should be grasped by anyone with a fundamental understanding of the subject, but your methodology and or results should tread on new ground. That is the impression I got watching the lecture. 2 things in particular blew my mind:

- Derivation of statistical bounds using optimization (min-max) and stability theory
- Using variational analysis of continuous functions to obtain more general forms of limited heuristic solutions

At some points I felt like I needed to download the papers/references noted by Professor Jordan and read them to fully understand his methodology, but I never felt like he didn't fully understand the underlying concept. With some modification he could give this same lecture to a complete novice, and the novice would grasp what Professor Jordan was trying to do: proof of true mastery according to Feynman.

I also felt (and still feel) like an utter dummy after watching the lecture. How does one get to that point?
Click to expand...

That was the exact thing I said to myself when I finished the lecture. Btw, in case you didn't know, professor Jordan recommended this list of book to Machine Learning scientists:

http://www.statsblogs.com/2014/12/3...-suggested-by-michael-i-jordan-from-berkeley/

I have started to read Casella's book on statistics and Golub's book on linear algebra, but they are far from trivial, and the others are even more difficult (actually read the first 9 chapters or so of Cover and Cover book in Information Theory a couple of years ago). My bet is that professor Jordan has read too many mathematics during his life, that not he finds trivial stuff that other researchers in the field haven't ever heard about.

On a side note, he mentioned quite a lot the accelerated gradient descent developed from Nesterov a lifetime ago. It is a gem that the community in US and Europe had overlooked, until 2012 or so, when Ilya Sutskever (back then a PhD student of Geoffrey Hinton, now leading researcher at Open AI) refound it, and used it instead of the common momentum on neural networks. While at the moment other methods seems to be preferred (Adam which is essentially just RMSProp with momentum), Nesterov's momentum was state of the art on optimization for a couple of years or so. I think that a student of Ng developed an algorithm called NAdam which replaces momentum with Nesterov momentum in Adam, but I haven't ever used it. Funnily enough, that was done just as project for Ng's course in Stanford CS229.

Btw, applying for a summer school this year when Jordan is one of the lecturers (the other famous ones are Ghamarani, Leskoves, Schkolkopf and Salakhutdinov).
 
Last edited:
Revan said:
That was the exact thing I said to myself when I finished the lecture. Btw, in case you didn't know, professor Jordan recommended this list of book to Machine Learning scientists:

http://www.statsblogs.com/2014/12/3...-suggested-by-michael-i-jordan-from-berkeley/

I have started to read Casella's book on statistics and Golub's book on linear algebra, but they are far from trivial, and the others are even more difficult (actually read the first 9 chapters or so of Cover and Cover book in Information Theory a couple of years ago). My bet is that professor Jordan has read too many mathematics during his life, that not he finds trivial stuff that other researchers in the field haven't ever heard about.

On a side note, he mentioned quite a lot the accelerated gradient descent developed from Nesterov a lifetime ago. It is a gem that the community in US and Europe had overlooked, until 2012 or so, when Ilya Sutskever (back then a PhD student of Geoffrey Hinton, now leading researcher at Open AI) refound it, and used it instead of the common momentum on neural networks. While at the moment other methods seems to be preferred (Adam which is essentially just RMSProp with momentum), Nesterov's momentum was state of the art on optimization for a couple of years or so. I think that a student of Ng developed an algorithm called NAdam which replaces momentum with Nesterov momentum in Adam, but I haven't ever used it. Funnily enough, that was done just as project for Ng's course in Stanford CS229.

Btw, applying for a summer school this year when Jordan is one of the lecturers (the other famous ones are Ghamarani, Leskoves, Schkolkopf and Salakhutdinov).
Click to expand...

I have Casella's book in pdf I believe. I didn't go too deep into it, it was just a stepping stone to stochastic methods. He also wrote a book on Bayesian methods which was my intro to the subject. Excellent list by the way. Did you see his note about studying each book 3 times? :lol: It's a wonder and credit to the man that he hasn't been poached by some private lab in San Jose being paid millions + stock options to develop some mining mechanism.

I know of the steepest descent method, but I never heard of Nesterov or his method until a few days ago. Jordan equated it's importance to the FFT so I'll be catching up on that sometime. The summer school sounds excellent, even if it's tailored more for the academic side of things.
 
adexkola said:
I have Casella's book in pdf I believe. I didn't go too deep into it, it was just a stepping stone to stochastic methods. He also wrote a book on Bayesian methods which was my intro to the subject. Excellent list by the way. Did you see his note about studying each book 3 times? :lol: It's a wonder and credit to the man that he hasn't been poached by some private lab in San Jose being paid millions + stock options to develop some mining mechanism.

I know of the steepest descent method, but I never heard of Nesterov or his method until a few days ago. Jordan equated it's importance to the FFT so I'll be catching up on that sometime. The summer school sounds excellent, even if it's tailored more for the academic side of things.
Click to expand...
Yeah, Casella had an another book on Bayesian methods, which is also on Jordan's recommended list. Sadly, I don't know anything about Monte Carlo methods, and at the moment, I don't even have much time for that.

Yep, credit to him for not choosing to be a multimillionaire in a time when almost every leading researcher has joined the industry. From the very big names in US/Canada, I think that it is only him and Yoshua Bengio who have stayed in academia.
 
So AI in past 6 months has beaten Chess, Go and now a huge step it beats humans at poker
Poker is importatnt step as the information in this game isnt as specific as in Chess or Go.Looks like the age of machines is upon us.
How close are we now to AGI? I think in next 10 years we will see human kind do some amazing things.
 
Maradona10 said:
So AI in past 6 months has beaten Chess, Go and now a huge step it beats humans at poker
Poker is importatnt step as the information in this game isnt as specific as in Chess or Go.Looks like the age of machines is upon us.
How close are we now to AGI? I think in next 10 years we will see human kind do some amazing things.
Click to expand...
Poker is a strange one though, the human players would be somewhat handicapped by not being able to read the physical signals they would get with human players. Its a similar issue to trying to compete in an online tournament versus playing around a kitchen table with your mates. Assuming its the article mentioned below, the players were also sharing notes and the computer was doing upgrades based on the gaps the players idetnified, makes it seem more like an exhibition to make the AI look good.

http://www.independent.co.uk/life-s...tware-beats-pro-players-victory-a7555791.html
 
Maradona10 said:
So AI in past 6 months has beaten Chess, Go and now a huge step it beats humans at poker
Poker is importatnt step as the information in this game isnt as specific as in Chess or Go.Looks like the age of machines is upon us.
How close are we now to AGI? I think in next 10 years we will see human kind do some amazing things.
Click to expand...
It has beaten Kasparov in chess 20 years ago, Seedol in Go a year ago and now humans in poker.

If AGI is hard-AI or general purpose AI, I think that at the moment we are not close, and as far as I am aware, there is no much research on it, mostly cause we have no idea how we could make it work. But soft AI is getting better each day.
 
On the topic of AGI, I watched an interesting video recently:



The stop button problem they describe goes quite nicely to show just how difficult it is to make a proper AGI.

Plenty more videos on the topic of AI/AGI for those interested btw.
 
Currently writing a Machine Learning algorithm for the last bit of AI coursework on my module. It's a basic specification, I can't even comprehend how complex the algorithms for AI that can win at Go or Poker are.
 
Maradona10 said:
So AI in past 6 months has beaten Chess, Go and now a huge step it beats humans at poker
Poker is importatnt step as the information in this game isnt as specific as in Chess or Go.Looks like the age of machines is upon us.
How close are we now to AGI? I think in next 10 years we will see human kind do some amazing things.
Click to expand...

To be fair, it doesn't really sound like it would be too hard to beat a human in poker. Today's poker is mostly maths, that by playing percentages and calculations correctly then over the course of your career you are expected to profit even if you lose big pots by making the right call mathematically. Any computer can be made to firstly make sure it makes the correct call based on the maths available which a human can't really do as exhaustion takes a toll and then just tweaked from there to include aspects of reading patterns made by your opponent and reacting to them.
 
@Revan

I don't suppose you have any links to some good classification data sets to test my ML algorithm against? If they come with an idea of classification accuracy achieved by other algorithms that would be even better :D
 
Pexbo said:
@Revan

I don't suppose you have any links to some good classification data sets to test my ML algorithm against? If they come with an idea of classification accuracy achieved by other algorithms that would be even better :D
Click to expand...
I don't have anything ;)

If your algorithm are feature based, then here you can find a ton of datasets: http://archive.ics.uci.edu/ml/

If you are tackling computer vision problems (most likely using deep learning approaches), then the datasets you might want to try are MNIST (http://yann.lecun.com/exdb/mnist/), CIFAR10 (https://www.cs.toronto.edu/~kriz/cifar.html) and ImageNet (http://www.image-net.org/). ImageNet requires to open an account, and it is massive 1T+. Bear in mind that most of deep learning libraries might have already MNIST and CIFAR-10, in addition to neural nets already trained in ImageNet. Also, scikit-learn might have some of the datasets that are in the first link I gave you.

For acoustic datasets you might want to use TIMIT: https://catalog.ldc.upenn.edu/ldc93s1

...

About accuracy, I am not really sure, but there might be something on the first link I gave you, while for the computer vision datsets, the accuracy is basically even better than for humans, typically achieved by using Residual Nets or Inception CNNs. I think that on MNIST the accuracy is 99.8%, on CIFAR-10 over 90% and in ImageNet around 97%.

...

An another good resource for datasets (and many other things) is Kaggle: https://www.kaggle.com/competitions
 
Revan said:
I don't have anything ;)

If your algorithm are feature based, then here you can find a ton of datasets: http://archive.ics.uci.edu/ml/

If you are tackling computer vision problems (most likely using deep learning approaches), then the datasets you might want to try are MNIST (http://yann.lecun.com/exdb/mnist/), CIFAR10 (https://www.cs.toronto.edu/~kriz/cifar.html) and ImageNet (http://www.image-net.org/). ImageNet requires to open an account, and it is massive 1T+. Bear in mind that most of deep learning libraries might have already MNIST and CIFAR-10, in addition to neural nets already trained in ImageNet. Also, scikit-learn might have some of the datasets that are in the first link I gave you.

For acoustic datasets you might want to use TIMIT: https://catalog.ldc.upenn.edu/ldc93s1

...

About accuracy, I am not really sure, but there might be something on the first link I gave you, while for the computer vision datsets, the accuracy is basically even better than for humans, typically achieved by using Residual Nets or Inception CNNs. I think that on MNIST the accuracy is 99.8%, on CIFAR-10 over 90% and in ImageNet around 97%.

...

An another good resource for datasets (and many other things) is Kaggle: https://www.kaggle.com/competitions
Click to expand...


Thanks man that's awesome. Just what I was hoping for.
 
@Revan Did you read this paper: https://arxiv.org/pdf/1701.08734.pdf

It sounds really interesting and important, but I am not sure what to make of it in the grand scheme of things, because I am a layman when it comes to these issues:


ABSTRACT
For arti cial general intelligence (AGI) it would be ecient
if multiple users trained the same giant neural network, per-
mitting parameter reuse, without catastrophic forgetting.
PathNet is a rst step in this direction. It is a neural net-
work algorithm that uses agents embedded in the neural net-
work whose task is to discover which parts of the network to
re-use for new tasks. Agents are pathways (views) through
the network which determine the subset of parameters that
are used and updated by the forwards and backwards passes
of the backpropogation algorithm. During learning, a tour-
nament selection genetic algorithm is used to select path-
ways through the neural network for replication and muta-
tion. Pathway tness is the performance of that pathway
measured according to a cost function. We demonstrate
successful transfer learning; xing the parameters along a
path learned on task A and re-evolving a new population
of paths for task B, allows task B to be learned faster than
it could be learned from scratch or after ne-tuning. Paths
evolved on task B re-use parts of the optimal path evolved
on task A. Positive transfer was demonstrated for binary
MNIST, CIFAR, and SVHN supervised learning classi ca-
tion tasks, and a set of Atari and Labyrinth reinforcement
learning tasks, suggesting PathNets have general applicabil-
ity for neural network training. Finally, PathNet also signif-
icantly improves the robustness to hyperparameter choices
of a parallel asynchronous reinforcement learning algorithm
(A3C)
Click to expand...
 
PedroMendez said:
@Revan Did you read this paper: https://arxiv.org/pdf/1701.08734.pdf

It sounds really interesting and important, but I am not sure what to make of it in the grand scheme of things, because I am a layman when it comes to these issues:
Click to expand...
I haven't read it, though the abstract seems quite interesting.

I will probably read it in the next few days, seems very interesting to combine deep learning with genetic algorithms.
 
@Revan thanks for the links, just uploaded my algorithm to the marking server and got 100% marks! (Had to be within 2.5% of a target accuracy for 3 different data sets). I'm going to have a crack at one of those Kaggle challenges over the summer, I've got my eye on digit recogniser.
 
Pexbo said:
@Revan thanks for the links, just uploaded my algorithm to the marking server and got 100% marks! (Had to be within 2.5% of a target accuracy for 3 different data sets). I'm going to have a crack at one of those Kaggle challenges over the summer, I've got my eye on digit recogniser.
Click to expand...
Digit recognizer is nice, but a bit boring. In order to do well there, you will need to use a CNN, and while it is easy to use it (especially if you use some deep learning library), you won't learn there that much, and you will achieve easily 99.5% accuracy.
 
JosDeVos said:
On the topic of AGI, I watched an interesting video recently:



The stop button problem they describe goes quite nicely to show just how difficult it is to make a proper AGI.

Plenty more videos on the topic of AI/AGI for those interested btw.
Click to expand...


That is a great video. It perfectly shows how complex programming even a basic instruction is.
 
https://homes.cs.washington.edu/~yejin/Papers/emnlp16_sonnet.pdf

a fairly robust program, that can write poetry.
843.png
 
Pogue Mahone said:
And so it begins...

I, for one, welcome our robot overlords.
Click to expand...

Jaysus. Just thinking, they'll eventually be able to develop a language that evolves so quick we won't have time to work it out, won't they?
And if they develop the ability to modify themselves (aka we're screwed), I imagine they could work something out that we can't even detect.
 
The notion of AI is so amazingly intriguing, yet terrifying. I cant wait to see how it all plays out.
 
After thinking about a bunch of old blokes pissing about in two conference rooms in different countries, discussing whether to do something that could kill thousands-millions, with one of these sides being voted in by a bunch of people that must've evolved from plankton, I'm starting to think the only chance of Earth working for a long time to come, and eventually in peace, is for something more intelligent to orchestrate things. Hopefully they're nice.
 
I am going to submit today my first CVPR (most important computer vision conference) paper. Very happy!

Somehow in the paper, there is also one of the most important researchers in the field of deep learning. Cannot give names for now, but if this paper gets accepted, not only I will have a paper in a top conference, but I will be a co-author of one of the deep learning fathers.
 
Revan said:
I am going to submit today my first CVPR (most important computer vision conference) paper. Very happy!

Somehow in the paper, there is also one of the most important researchers in the field of deep learning. Cannot give names for now, but if this paper gets accepted, not only I will have a paper in a top conference, but I will be a co-author of one of the deep learning fathers.
Click to expand...

Congratulations!

Is the Caf cited anywhere?
 
Revan said:
I am going to submit today my first CVPR (most important computer vision conference) paper. Very happy!

Somehow in the paper, there is also one of the most important researchers in the field of deep learning. Cannot give names for now, but if this paper gets accepted, not only I will have a paper in a top conference, but I will be a co-author of one of the deep learning fathers.
Click to expand...

That's incredible mate, keep us updated, I'd love to have a read if you get it published. What angle have you been researching it from?
 
Revan said:
I am going to submit today my first CVPR (most important computer vision conference) paper. Very happy!

Somehow in the paper, there is also one of the most important researchers in the field of deep learning. Cannot give names for now, but if this paper gets accepted, not only I will have a paper in a top conference, but I will be a co-author of one of the deep learning fathers.
Click to expand...

Well done! Hopefully your paper gets accepted :)
 
adexkola said:
Congratulations!

Is the Caf cited anywhere?
Click to expand...
Thanks. Of course that I've cited Caf.
Pexbo said:
That's incredible mate, keep us updated, I'd love to have a read if you get it published. What angle have you been researching it from?
Click to expand...
This is just a dataset actually, with a simple CNN based baseline and some directions. It got created because we were working on a problem which didn't have a suitable dataset.

I will obviously mention it here in case it gets accepted (I think that CVPR results are in February).
AXVnee7 said:
Well done! Hopefully your paper gets accepted :)
Click to expand...
Thanks! I hope it too. It is actually strange considering that I've worked only a couple of months in this and we're going for the top computer vision conference, while I worked on 3 failed attempts so far much more and we couldn't even make a submission (the results were crap).
 
Revan said:
I am going to submit today my first CVPR (most important computer vision conference) paper. Very happy!

Somehow in the paper, there is also one of the most important researchers in the field of deep learning. Cannot give names for now, but if this paper gets accepted, not only I will have a paper in a top conference, but I will be a co-author of one of the deep learning fathers.
Click to expand...
Wow. That's impressive. Hope your paper gets accepted. :)
You left it pretty late, didn't you? Today is the deadline. A couple of friends of mine also submitted, but last week.

I am looking forward to ACL and EMNLP next year myself, papers underway.
 
Gandalf Greyhame said:
Wow. That's impressive. Hope your paper gets accepted. :)
You left it pretty late, didn't you? Today is the deadline. A couple of friends of mine also submitted, but last week.

I am looking forward to ACL and EMNLP next year myself, papers underway.
Click to expand...
Yeah, it is a paper from a work I did in an another lab, so I was prioritizing some other project (while the two guys there were quite busy with another project of them). We had an internal deadline 2 weeks ago, but ICCV screwed everything, getting all of my time for 10 days, so this had to be pushed. Being a dataset though, I think that it has a good chance of being accepted, and if not, then we would have two weeks to just refine the paper before submitting to ECCV. I mean, the work is what it is (it is good IMO), so all depends on how well the paper is written (not great IMO).

After this, plan is to go back to the other project I was working, and if we have something good, to go for ECCV/ICML, otherwise ICPR or something lower just to close the project. I am very skeptical of the project, and don't believe at it.

Then the dream is to have something for NIPS (NIPS beats everything IMO). I have some ideas and I love to have something for NIPS, especially considering that it will be the last NIPS before I finish the PhD and so I want to do some job hunting there (this year I will be in NIPS, but without paper, and not caring that much for job hunting considering that I have another 22 months in the contract of PhD).

PS: There have been over 3000 submissions to CVPR this year. Machine Learning/Computer Vision has gone crazy in these years, every conference has significantly more submissions than the previous one. NIPS tickets were sold out 3 months before the conference is starting, that shit doesn't happen even for concerts :lol:
 
You must log in or register to reply here.