Machine Learning, anyone here know about it?
Discussion
Looking for a little advice really,
I need to do some training for matlab and I think I am going to go on their statistical methods course. The Machine Learning course is schedule for the 2 days after that, so I am trying to work out if I actually need to go on it and whether it will be useful because these courses are £1100 each!
So the data I will be working with will be quite large. There will be lots of vehicle data and it will be in the form of summaries (i.e. day, week, month, trip) which will be comprised of all kinds of things; averages for speed, fuel used, payload. Engine speeds might be in the form of averages and distributions (i.e. time spent in each engine speed band) and heat maps (speed-load) etc. We will also have raw data streams of trips, second by second data, that will be processed down into averages and portions of the trip categorized into urban, rural, motorway etc. There will be weather data also.
So lots of different types of data
And I need to be looking at not just providing statistical outputs (average speeds, fuel use, standard deviations and what not) I really need to be looking at parameters that are linked or have influence on one another, find any patterns...
i.e. I need to be able to answer, "why has such and such vehicle performed poorly" ... I should be able to, from my analysis show why. I am envisaging something like:
Poor performance because:
The wind speed was high and predominantly in a particular direction, your vehicle spent x% time traveling against the wind. There was also an average 3 mm precipitation and the standard deviation of vehicle speed shows the driving might have been erratic. Here are some graphs that show the relationship of these things to performance; you need to get your driver to be more consistent with his speed and avoid driving in heavy rain etc etc.
Essentially I need to derive sets of conditions and relationships that effect vehicle performance. I could then even model an outcome if some known or predicited conditions are fed in one end, get some kind of vehicle performance parameter (deviation from rated performance or something) out the other side?
Machine Learning seems to be everywhere and a bit of a buzz word. I don't want to fall into the trap of getting all excited about it and then not applying it correctly. Then again, its training, paid for by the company and I should probably just have it!
I need to do some training for matlab and I think I am going to go on their statistical methods course. The Machine Learning course is schedule for the 2 days after that, so I am trying to work out if I actually need to go on it and whether it will be useful because these courses are £1100 each!
So the data I will be working with will be quite large. There will be lots of vehicle data and it will be in the form of summaries (i.e. day, week, month, trip) which will be comprised of all kinds of things; averages for speed, fuel used, payload. Engine speeds might be in the form of averages and distributions (i.e. time spent in each engine speed band) and heat maps (speed-load) etc. We will also have raw data streams of trips, second by second data, that will be processed down into averages and portions of the trip categorized into urban, rural, motorway etc. There will be weather data also.
So lots of different types of data
And I need to be looking at not just providing statistical outputs (average speeds, fuel use, standard deviations and what not) I really need to be looking at parameters that are linked or have influence on one another, find any patterns...
i.e. I need to be able to answer, "why has such and such vehicle performed poorly" ... I should be able to, from my analysis show why. I am envisaging something like:
Poor performance because:
The wind speed was high and predominantly in a particular direction, your vehicle spent x% time traveling against the wind. There was also an average 3 mm precipitation and the standard deviation of vehicle speed shows the driving might have been erratic. Here are some graphs that show the relationship of these things to performance; you need to get your driver to be more consistent with his speed and avoid driving in heavy rain etc etc.
Essentially I need to derive sets of conditions and relationships that effect vehicle performance. I could then even model an outcome if some known or predicited conditions are fed in one end, get some kind of vehicle performance parameter (deviation from rated performance or something) out the other side?
Machine Learning seems to be everywhere and a bit of a buzz word. I don't want to fall into the trap of getting all excited about it and then not applying it correctly. Then again, its training, paid for by the company and I should probably just have it!
rustyuk said:
We are currently using the MS Azure Machine Learning Studio. Lots of free traning available online.
Will check it out. We use Matlab mostly as we have lots of models for vehicles and batteries etc in either matlab scripts or simulink. I have also had a go with Orange which is free and is a application for Python. It seems really nice to be honest, but the integration with the rest of matlab wouldn't be there and their examples/tutorials aren't that great.ML is a very broad area and like many good buzzwords the reality is somewhat less exciting than the promise.
99% of the time ML translates into applying supervised learning, where you have an labelled dataset with which to train an algorithm. In simple terms the learning set contains the correct output (answers) you are looking for given the inputs. You run the data set through the algorithm and see how close it is to the correct answers, then rinse and repeat until happy. Once you have the final algo then you can use it on un-labelled data of the same nature.
There are more esoteric forms of ML that now tend to get branded Deep Learning. For example the stuff that Deepmind do around the game Go and other things are a form of Reinforced learning for which you need a well defined reward function to allow the algorithm to learn.
From the description of your data/problem then it may be that some unsupervised learning approaches such as k-means clustering would be useful.
In short, go on the course as it won't harm your CV
Most of the Data Scientists I work with don't tend to use Matlab for this stuff though.
99% of the time ML translates into applying supervised learning, where you have an labelled dataset with which to train an algorithm. In simple terms the learning set contains the correct output (answers) you are looking for given the inputs. You run the data set through the algorithm and see how close it is to the correct answers, then rinse and repeat until happy. Once you have the final algo then you can use it on un-labelled data of the same nature.
There are more esoteric forms of ML that now tend to get branded Deep Learning. For example the stuff that Deepmind do around the game Go and other things are a form of Reinforced learning for which you need a well defined reward function to allow the algorithm to learn.
From the description of your data/problem then it may be that some unsupervised learning approaches such as k-means clustering would be useful.
In short, go on the course as it won't harm your CV
Most of the Data Scientists I work with don't tend to use Matlab for this stuff though.If you're already using Matlab or know people who are then I'd say stick with it, it's not cheap but it's industry standard and very powerful.
You can get the book and training material here (but I guess you know that already)
https://uk.mathworks.com/campaigns/products/ppc/go...
I've got them if you can't be bothered dealing with the spam after signing in and want to PM me.
I did the course a few years ago, my application was pretty simple and I didn't really but scrape the surface of what could be done. As with anything the tool is often much more powerful than most of the applications or people who would use it.
You can get the book and training material here (but I guess you know that already)
https://uk.mathworks.com/campaigns/products/ppc/go...
I've got them if you can't be bothered dealing with the spam after signing in and want to PM me.
I did the course a few years ago, my application was pretty simple and I didn't really but scrape the surface of what could be done. As with anything the tool is often much more powerful than most of the applications or people who would use it.
Thanks guys
To be honest I am not personally wedded to matlab. There is sentiment in the office that we should try ween off it really as its quite expensive to maintain and everything is in a tool box these. Its like buying an Audi.
I think the major item in ML is actually pre-processing the data into a form that is going to be useful, so I need to think about what I am going to end up with before I might want to start applying ML.
To be honest I am not personally wedded to matlab. There is sentiment in the office that we should try ween off it really as its quite expensive to maintain and everything is in a tool box these. Its like buying an Audi.
I think the major item in ML is actually pre-processing the data into a form that is going to be useful, so I need to think about what I am going to end up with before I might want to start applying ML.
FredClogs said:
If you're already using Matlab or know people who are then I'd say stick with it, it's not cheap but it's industry standard and very powerful.
You can get the book and training material here (but I guess you know that already)
https://uk.mathworks.com/campaigns/products/ppc/go...
I've got them if you can't be bothered dealing with the spam after signing in and want to PM me.
I did the course a few years ago, my application was pretty simple and I didn't really but scrape the surface of what could be done. As with anything the tool is often much more powerful than most of the applications or people who would use it.
Cheers FredClogs, I have those already somewhere. Not had the chance to read them mind. You can get the book and training material here (but I guess you know that already)
https://uk.mathworks.com/campaigns/products/ppc/go...
I've got them if you can't be bothered dealing with the spam after signing in and want to PM me.
I did the course a few years ago, my application was pretty simple and I didn't really but scrape the surface of what could be done. As with anything the tool is often much more powerful than most of the applications or people who would use it.
I second the suggestion to use something else - R is my preference.
There's an introductory machine learning course on Coursera taught by Andrew Ng (https://www.coursera.org/learn/machine-learning). It uses matlab for it's coursework exercises. It's free and, if you're a ML newbie, I suspect better than what matlab specific training will offer.
There's an introductory machine learning course on Coursera taught by Andrew Ng (https://www.coursera.org/learn/machine-learning). It uses matlab for it's coursework exercises. It's free and, if you're a ML newbie, I suspect better than what matlab specific training will offer.
nammynake said:
The problem you're describing sounds like it would be best addressed with 'standard' statistical analysis and modelling techniques (linear/logistic regression). SAS would be perfect for that.
Yes I had a feeling that would be the case. I might be wrong but from playing in Orange, ML has a 2 parts and the first part is the data cleaning and statistics analysis. The second part is then building and teaching models with that data. I think the first part is a must, but I was just interested to see if something could come of the ML part....... We already have vehicle models that we can run on whatever drive cycles we want. I was thinking the ML could provide a more macro model, a black box, where you just input the type of vehicle, its modal driving type and out comes some MPG and emissions data etc. Not a fully formed thought I admit, but just musing.
We are not wedded to Matlab, as said we are looking to move to the more open source things, but we have Matlab licenses bought and paid for for the next year and we have a handful of people who can wield it decently so kinda makes sense to try stick with it.
Having said that..... I am not sure if we actually have a machine with the statistics tool box....
EDIT: We do have the stats and machine learning tool box.
Edited by Otispunkmeyer on Friday 9th June 11:39
http://news.efinancialcareers.com/uk-en/285249/mac...
Good overview of ML, and languages to use. Different industry, same principles.
Good overview of ML, and languages to use. Different industry, same principles.
Gassing Station | Science! | Top of Page | What's New | My Stuff