Implementing decision trees with Python

Last time , we explored the basic math foundation behind the decision tree algorithm and we went over concepts such as entropy and information gain; and how these ideas tie together in the ID3 algorithm to form the basics of decision trees. Now , we’re going to review a simple python implementation that demonstrates how the algorithm works at a basic level solving a simple classification problem.

The Data

In order to be able to track the results of the algorithm is always best to have a small toy dataset that makes it easier to visualize and understand the algorithm output. For this exercise , I’m going to use a very simple dataset taken from the Fundamentals of Machine Learning book (2015) that is set up specifically to test and ID3…

First Entry in Basics of Machine Learning

Decision trees are one of the most common and widely used machine learning algorithms and are particularly useful for predicting categorical (non numerical) variables , making them a powerful tool for Predictive Analytics.

Rather than relying in decision tables, that even for small toy datasets can run unmanageably large, decision trees lean in the concept of information entropy and construct its prediction path by increasingly choosing the path from which we can gain the most information.

So what does that mean?

The idea behind decision trees is to progressively select the most descriptive features that discriminate between observations that have different categorical values. To achieve is, the discriminatory power of a variable is estimated by analyzing the size and probability distribution of each set of observations, by estimating how pure is each set of instances with respect to the objective variable we’re trying to predict. …

Going beyond Iris-Setosa and the Titanic classification

One of the most fundamental , and indeed, most interesting aspects of Analytics is that is near omni-present across many economic sectors today. This ubiquity offers a great deal of opportunities , but also can be very confusing to those who want to get their foot in the door and become data analysts. Much is discussed about what it entails to be a Data Scientist, Data Engineer or Analyst or a Machine Learning Engineer, however is not often that these roles are described within the context of the industry they serve. …

How do enemies find us

I’ve been away for a few weeks doing tons of research for new materials that I can bring for this blog and thinking how best to organize them. Also, I’ve been designing a study and project plan to share my AI, Data Science and developer journey with the community and I believe. After much deliberating , I decided to split the blog into three main categories :

1. Development: we’re going to deal here with the specifics of creating new software projects , these articles will be code intensive and will verse on topics such as software projects, algorithms, design patterns and development tools and frameworks. …

AI and medicine applications

It is no news that the AI has started to permeate virtually all aspects of modern life , in often unexpected ways. We all know about Netflix recommendation algorithm or the data mining Facebook performs on user´s preference data(one of may reasons I only visit my Facebook account about once a year), and that´s all very exciting, but in my opinion the real value of Machine Learning (ML) and AI is to try and find applications that solve problems in different domain specific fields. …

Python is truly a versatile language. While I mostly use it as a study platform to learn machine learning, analytics and artificial intelligence, that doesn’t mean that Python isn’t a fully fledged developer language on its own and from time to time I like to build small things with it while I have a longer-term analytics projects in the works.

The product I brought you today is one such project, as an enthusiast of Role Playing Games (RPG’s), I often find myself wanting to have a way to simulate or test a character I may be building and maybe test him out on his combat skills. …

In my last entry , I discussed the ins and outs of my Naive Bayes Algorithm. This week we talk about the last stage in the Analytics Project, which is the result and their analysis and what can we learn from the observations made.

First, a quick reminder on where we left or dataset last week . The dataset had a total of 1'834'517 entries for the 3 crimes selected for study (mugging, rape and murder). The crime instances registered within the dataset were distributed as follows

My first entry enjoyed good acceptance on the people who read it , but the common theme was to ask about the “how” as much as the “what”. When it gets down it , people wants to know the nitty-gritty details of the code that develops an analytics projects.

To me code is important , no doubt about it, however I decided to start my medium blog with the concepts of a machine learning model which, to me, are far more important than the code procedurals, after all I can´t code anything without first understanding what it is what I´m trying to solve. Sometimes, this lack of “basis-first” approach can be seen in some educational materials which prioritize throwing a bunch of pandas , numpy or pytorch code at your without explaining what it means. It might as well be written in ancient Sanskrit for all the help is going to be . …