Friday, March 25, 2016

The most important skills for a Data Scientist

This post has been written as homework for the course
"Statistical Thinking for Data Science and Analytics"
which I'm attending through the edX platform
at Columbia University.

Question:
What are the most important skills for a Data Scientist?

In a short list:

·         Statistics & Probability
·         IT
o   Databases
o   Big Data
o   Cloud Computing
o   Graph Design
o   Data Mining
o   Machine Learning
·         Insight
·         Ability to team with people specialized in different kinds of activities

Statistics and Probability

A Data Scientist shall know enough Statistics to be able to find what is meaningful in the collected data points and its correlations and what is just “noise”.
Actually, to start, a Data Scientist needs to be able to assess, in a collected dataset, the “margin of error” (“sample error”), a possible bias (“sample bias” and – last but not least – what is missing in the sample, since “N = All” is, almost certainly, not true.

IT

“Big data analysis” will require advances in statistical methods – and will require advances in IT, as well.  But just as the new resources in statistics shall be built on the existing statistical knowledge, the (relatively) new tools to deal with (mostly Big) Data from several sources, like the “Web Exhaust” data stream, genomic projects, astrophysics, and so on, shall be better understood and used when we take in account the existing database management tools.
A Data Scientist shall master relational databases and NoSQL databases as well as Data warehouses.   Needs to be able to create data presentation in forms that allow himself to understand the data and find patterns and create hypothesis and also communicate his findings to non-specialized audiences. 
And, to be able to use these tools in massive datasets, will need to be familiar with Cloud Computing, IaaS and SaaS, as can be used, for example, in AWS.  

Insight

A Data Scientist needs to provide the insight, the inferring which sometimes the “black-box algorithms” of Machine Learning fail to achieve.  To be able to put apart the “signal” from the “noise” among a huge amount of possible correlations, and point out the “causality” among these “correlations”.

Ability to Team

Since Data Science is essentially multidisciplinary, a Data Scientist needs to be able to team up, learn by himself and quickly, understand the domain of the project he is working, and communicate efficiently.



Thursday, January 15, 2015

Big Data – Comments from a Beginner

Big Data – Comments from a Beginner
Pedro Francisco Borges Pereira
Undergraduate Student – Universidade Luterana do Brasil (Lutheran University of Brazil)
Major – Information Systems
International Exchange Student at Kansas State University
Fall/2014 Semester

Foreword

 I was very interested in Big Data – is there some IT professional who is not, nowadays? – and decided to learn about. So I´ve enrolled in, and I´m attending, the CIS 798-Programming Techniques for Big Data Analytics, at Kansas State University, with Professor William Hsu.
I started the course aware of some catchwords, like
“Big data is NoSQL”.
“Big data made relational databases obsolete”.

Professor Hsu started the course teaching us that “Big data is about Volume, Velocity, Variety”, and proceeded with a Machine Lab – implementing the prototypical Word Count Big Data programming example, using Hadoop with a Java plug-in.
Well, that makes sense, but I think Professor Hsu underestimated my lack of knowledge.  I was not just ignorant regarding the programming techniques; I did not know when, in which use cases, to apply Big Data programming techniques.

So I thought I should take some time to learn about the context, about in what use cases we could and/or should use Big Data techniques and also, if “traditional” relational databases are not dead, when we should stay with them. 

As a result of my knowledge about Big Data (or, more properly, lack of) summing up almost just to the catchword “Big Data is NoSQL”, when I speak, in these initial paragraphs, about “Big Data infrastructure”, please understand that I mean “anything different from relational databases”.
I called myself a “beginner”.  Indeed I am – regarding Big Data.  I´m familiar (proficient, I would dare to say) with relational databases.  I think these notes can be useful to anyone aiming to get a general overview of Big Data, but please take in account that I´m writing as somebody used to relational databases and new to Big Data.  I did make a reasonable amount of reading about Big Data, but of course I am still a beginner.  I may have made mistakes; I very probably did. Your feedback will be very welcomed at my personal e-mail:  pedrofbpereira@yahoo.com.br .  Thank you very much!


Initial Statements

 I´ve read a dozen articles (see Bibliography, in these Notes), as well as Professor Hsu handouts, and reached, in short, the following conclusions:
·         Despite the fact that (understandably) some NoSQL database vendors say so, not all applications using relational databases worth being converted to Big Data techniques; relational databases – and data warehouses using non-normalized “star schemas” and cubes - are still, frequently, the best option.
·         Despite the fact that some relational databases vendors (understandably) state that their products can do everything that Big Data frameworks do, they can´t. They can indeed deliver some Big Data tools. But amazing new fields – as for example Complex Event Processing - are being pioneered thanks to Big Data techniques.
·         “Volume, Velocity and Variety” is a simple way to describe Big Data features to outsiders.  Reality is not always that simple.  Actually most relational databases can scale to “big data” volume (usually, at a higher cost; usually, not instantly) and can perform at high speed (depending on the use case, faster than Big Data frameworks).  “Variety” would be the key feature.   I´d like to highlight that, as I see, this “variety” is more relevant in a sense of “changes in database schema” (or even absence of schema) instead of variety of data sources or in data content.
·         Economic differences and infrastructure differences:
o   A strong point of Big Data frameworks is distributed processing and storage, which leads to great easiness in scalability and failure tolerance.   Relational databases can be distributed and grow to sizes that would grant them the right to be called also “big data”.  But since Big Data frameworks are based in “standard” cheap computers, and relational databases uses (mostly) expensive powerful servers, distributing “conventional” relational databases is a lot more expensive.  Big Data frameworks tend to be cheaper.
o   Big Data frameworks can scale instantly.  Relational databases, usually, are not able to answer instantly to explosive demand increases.
o   Relational databases “can” be hosted in the cloud.  Most Big Data frameworks are born in the cloud.  Open source frameworks as for example Apache Hadoop are less expensive than “cloud” versions of relational databases.  And big providers of Infrastructure as a Service – as for example Amazon – allow using a safe, best-practices-managed, and highly (and instantly!) scalable platform, for an affordable cost.
·         Giants as Google, Yahoo, Amazon, the government, and also academic researchers, almost for sure need to use Big Data frameworks.  Smaller companies – thanks to IaaS providers – can also use Big Data.
·         There is no free lunch!  The increased capability of Big Data frameworks to deal with variable schemas implies in not being ACID[1] (Atomic, Consistent, Isolated, Durable).  Despite some NoSQL distributors claim to be ACID, I honestly can´t understand how you can be distributed, un-related, un-locked and, at the same time, ACID.  They aim to be, instead, BASE[2] (Basically Available, Soft-state and Eventually consistent).



[1] This performance/ACID exchange is clearly stated in “Dynamo: Amazon’s Highly Available Key-value Store” article, at http://www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf .
[2] See “Enterprise NoSQL for Dummies”, MarkLogic Special Edition, by Charlie Brooks. 


Contents
Foreword 1
Initial Statements 4
For some Use Cases, Relational databases are better 5
Big Data does things that Relational Databases can´t 5
Un-Structured Data 5
Complex Event Processing 6
Sessionization 6
Volume, Velocity, Variety 6
Volume 6
Velocity 7
Volume, Velocity – Facebook 7
Variety 7
NoSQL 7
Scheme-agnostic, but organized 8
Key-Value Stores 8
Column Family 8
Document Databases 8
Graph Databases 8
Big Table and HBase 9
Infrastructure Characteristics 10
Distributed, Parallel and Redundant – High Performance and Availability 10
Super-Computing / Cloud Computing 10
Hadoop Distributed File System - HDFS 11
MapReduce 12
The MapReduce Pipeline 12
Hadoop – Simple Definition 13
Hadoop – BEOCAT instance 13
AMAZON WEB SERVICES – CLOUD COMPUTING 14
The Canonical Wordcount Example 16
Environment: Linux;   Programming Language: Python 16
Word Count Diagram 17
hadoop-streaming.jar 17
Mapper 18
Reducer 18
Our Mapper and Reducer can run just in Linux 19
Running our Hadoop Job 20
Using Parallel Processing 20
BIBLIOGRAPHY 21
Glossary 23
Appendix – a BigData Solution Architeture 24
Felipe Renz MBA Project at Unisinos College, Brazil 24


Wednesday, December 17, 2014

Some comments about Agile Methodology

Agile Methodology – Some comments

I couldn´t agree anymore with Agile basic concepts, as summarized in the Agile Manifesto ( http://agilemanifesto.org/ ).
But I think that some Agile Developers underestimate the value of Software Engineering tools, mostly Business Process diagrams and UML diagrams.  I think that this could – sometimes should – be used not as “documentation”, as something built after the software has been built, but as “project” – something that will guide the development!  To me, it makes as many sense to start to code without a project, as it would be to start to build a house without a plan!

On the other hand, the fact that Agile Methodology clearly states that “it´s impossible to collect and specify all requirements” is a remarkable insight.  This fact must be accepted and all software developers must learn how to deal with it.  It´s up to the experienced Systems Analyst to decide when enough requirements has been defined, when the essence of the application has been defined - and start developing, leaving accessory features to be detailed later.

The concept of “frequent deliveries” is also awesome.  It keeps the development team focused, maintain users engaged, build a strong link between developers and users and avoid deviations from the actual goals.
Some tools we learned to use – backlogs and charts, for example – are a perfect support for this “frequent delivery” approach.  I´m already using this in other courses´ projects.

The auto-managed team, working physically close and constantly sharing issues and achievements, is a strong point of Agile methodology.  This does not leads “just” to better software – it results in more responsibility over team member´s shoulders - and in more freedom, more fulfillment, in a better and happier professional life!

Regarding the concept of “multidisciplinary teams”:  I agree that a developer must know how to test; that a database designer shall know how to build a user interface – and vice-versa.   And I agree that a developer must be able to use more than a single language and/or environment development.
But I think that this can lead to a team composed by “jack-of-all-trades” technicians, who know a little bit of many things but not much of any.  I do believe that in order to build good applications, good information systems, you do need specialists.  People who deeply know all features of the tools you are using – database engine, web servers, programming language…  In my opinion, none of the specialists shall live in a “bubble”, knowledge can and shall be transferred.  But a team needs highly qualified specialists.

I´ll dare to express some personal opinions:
According DZone[1], “In 1994, Standish reported a shocking 16 percent project success rate, another 53 percent of the projects where challenged and 31 percent failed outright”.
This is really shocking.  Are we developers that bad?  I would say that, in average, yes!  Some of our failures are poor or incorrect user requirements survey and specification, poor design, lack of technical specifications and – last but not least – lack of knowledge of the development tools.  Some developers even claim that using software engineering tools is useless, just a loss of time, because even using them the software will still be flawed.  Yes, when they tried, this happened – because they don´t know how to use software engineering tools correctly.  The issue is not in the tool, not in the methodology – is in the lack of knowledge of the developer!
NOTE:  you can see also "The Standish Group Chaos Report" ( http://www.projectsmart.co.uk/docs/chaos-report.pdf ) for more about software projects.

I would say that, regarding development tools, we are better.  Most of us are good coders.  But some current management concepts are also undermining this.
The truism that “who is able to code in one language can learn in any other in two weeks” is repeated as a truth evident in itself.  It can be true if – and this is a big “if” – the language paradigm is the same.  If not, yes, you can learn to code “something” in two weeks.  Something that will probably fit in the 85% of “challenged” or “failed” projects. 

Again – we do need specialists, we do need mastering our tools!  You can work with Oracle, or SQL Server, or C#, or ASP.Net, or MVC .Net, for years, and do not know 100%, not even 80%, of these tools´ resources.  You can´t call a two weeks experiment “learning” something like these!  A college course, one semester using the tool – that´s a beginning!  That´s when you are ready to start to use it professionally!

In my personal experience, mostly with some younger colleagues (yes, there are exceptions, some young developers are amazingly proficient in the technologies they use, I´m just talking about the “average”, the “majority”, that I´ve seem) are able to build a new application´s framework very quickly.  And then they got stuck.  Their knowledge is too superficial to enable them to develop the more sophisticated requirements.  Using again the analogy with building houses – it´s easy to build the walls, the roof.  But you can´t live in a house without the electric wires, the plumbing…  With software, sometimes our users have to live with light of lanterns and buckets of water.
Even when things do work, frequently the developer is not aware, or is not able to explain, how it works!  This is not a comfortable situation! I don´t think this is acceptable! Yes, software development has a component of “art”, “creativity”, but it is also science, it´s engineering, and an engineer does not “guesses” nor "hopes", an engineer “knows”! 

The Solution?

I think we shall adhere to Agile principles, but reverse the tendency (understandable tendency, a turn-around from extremely rigid "waterfall" was needed) to reject software engineering tools.
We should also select one or two developers in each team to be “the guys” for each requirement – database, web security, business process survey, etc.  Everyone must be able to do the “elementary” work in every area – but we need Subject-Matter Experts.
We should respect our tools.  Before we start to build a new project with a new programming language or development tool, we need to read about the basic concepts, about the paradigm.  We should meet – physically or virtually – some experts in this new tool.  We should try to develop something simple with the new tool – preferably something which we already developed with one of our favorite “old” tools.  Then, maybe, we´ll be able to start some “real life” development.

A last word, about maintenance.
I think that in some few cases, it´s evident what is wrong and (in even fewer cases) the error is very isolated.  Those are the (rare) cases when it is quick and easy fixing something, even if we are not familiar with the application.
I think that, as a general rule, in order to maintain an application which with we are not familiar, we should act as physicians – “primum non nocere” , “first, do not harm”.  And to “do not harm”, we need to understand the application.  If it is not documented, we should create some documentation.  We should try to understand how it has been built, what the developers who did build it had in mind. 
And, of course, we most domain the language (and/or framework) which the application has been built with.
Without these precautions, we would be not software developers doing maintenance, we would be just agents of the entropy!




Saturday, December 13, 2014

WHERE DO WE GO FROM HERE

WHERE DO WE GO FROM HERE

The semester is ending, and the last entry in this blog – at least the last requested by our teacher – is regarding “Where do I go from Here”.  Quoting literally: “What would your dream job in Computing Science be? What topic would you like to learn more about? What classes are you looking forward to taking? Are there areas that you wish this class would have covered more deeply?”

Well, it happened that I spent lunchtime chatting with three of my colleagues, undergraduate students.   One from Computing Sciences, the other two from Veterinary and Agronomy.  A very interesting chat, which completely changed what I had in mind to write this article.  Beginning with the title, “I” became “we”.

At our lunch meeting, all of us, except one, agreed that machines are capable of learning, act intelligently and make decisions, self-preservation and auto-repair, build another machines…
The discordant voice argued against machine learning and intelligence, which he thinks would be just a pathetic mockery of human mind’s capabilities.

I then presented some of the ideas I read in Erich Fromm’s books[1] (at least, as I understood it) – that a significant share of human beings do live, also, without exercising their learning capabilities and intelligence.  I think that computers are not – still – able to compete with human geniuses, but they already are, in several areas, better than the “average” human.

After this conversation, I started to think about some points, some beliefs, some understandings, I have:
·         Just as a book is a record of the author’s thoughts and feelings, a software is also a record of its author(s) personality traits.  The software records, usually, a narrower area from the authors’ personality than a book, but, even so, no few times, it reflects not just “logical reasoning”, it is shaped also by emotions and world vision.
·         Unlike a book, a computer acts and interacts with the physical world.  In that sense, a computer is a “being”.
·         We humans are the computer’s “Gods”.  We created them according to our likeness. 
·         The likeness is imperfect.  But not, as my colleague said, regarding “intelligence”, or even “creativity”[2].   No, I think that we can’t “imprint” in our technologic creations is what we can’t explain about ourselves!  I think we could use the word “soul”, but not in a theological sense[3].  I mean here, by “soul”, our emotions, our sub-conscious, our existential doubts.  We can’t program ethics, or philosophy, because we don’t know “the truth”, or “a safe way to the truth”, in these areas.
·         Computers – I mean hardware and software – soon will be reproducing themselves.  Creating new hardware and also new software – software perhaps, very likely, impossible to understand for us humans.  Then they will have become a new “species”.
·         We will share the world with this new species.  What are our future prospects, if our super-powerful and soul-less creatures escape our control?

Furthermore: I do believe that this “future” is happening right now!
Supercomputing thru parallel processing, interconnectivity, internet of things, artificial intelligence – all this is going to change the world, change radically, and only one thing we can take for certain  –it is that we can’t predict the results of these changes!

My dream?  Well, I want to be Sarah Connor!  :-)  
I’m kidding, just to try to hide my embarrassment with the nature of my ambition…
I want to learn about the human soul.  About ethics.  About happiness. 
Not in order to program computers in these areas.  No.  To keep them away from it!  To be able to clearly understand, and explain, and persuade people that we can’t delegate pursuit of happiness to computers!   Computers must be our tools, not our guides!
Can it be different in the future?  Of course, that’s what Science do, it moves forward, everything can change!   But I can’t see this changing, not in my lifetime.

Well, besides prevent humanity of being the slaves or the defenseless preys of machines[4], what do I dream as an IT professional?
I want to help using computers in the areas they can bring the greater good.  And I want to excel in my job, my art, I would call it.  Never stop learning and improving, be in the cutting edge, feeling that I'm helping to shape my profession – that would be wonderful!  I feel this like a kind of loving relationship with my science, with my art – because, as I said, I think that software development is not only engineering but has also a creative, artistic component.

Furthermore: I would like to leave a legacy behind my existence, a legacy carried out thru my professional capabilities.  I don’t want to be famous, not as an individual.  But I want to do something more meaningful than increasing the profits of some company.
I’d be so glad, so happy, if someday I do work in a team whose goal would be preservation of life, decrease of suffering!   Pain is part of life, is unavoidable, we need to accept it.  But, even so, every gesture that relieves pain is very meaningful, and very rewarding.
As I said: I don’t dream about being a hero, I don’t want to be famous.  I just would like to be able to die knowing that what I did for a living helped somebody, someday, being a little bit happier.

There’s another kind of legacy I’d like to leave.  I’d like to share whatever knowledge I have.  I already did this, not just as a programming instructor, but as a software developer.  For example, when running my own small software house, I had to guide young employees and interns.  More than one of them, several years after they left my company, meet me and said “thank you for helping me start my professional live, I learned a lot with you”.  I felt fulfilled!  And I feel that it’s not just about me and them, it’s a flow, and we are part of it!  They already shared their knowledge also. People who I did not know can be right now using some technique I showed my pupils – or even some technique my ex-pupils created based on what I showed them.  I don’t care if they don’t know my name; it’s a journey, we are passing on a torch, a flame; knowing that I did my part makes me happy.

“Oh my God, Pedro, can you please plainly say something about your practical and immediate plans regarding your professional life?  Instead of playing the “tavern philosopher?”"
Well, I can try!  :-)
I’m good with databases and with programming logic, and I like to work using this aptitudes.
So I think that what makes more sense to me is trying to learn and grow in the Data Science area.
Since I want to do something more meaningful than selling soaps, I will look for a job in an area linked to health, or safety, or environment.
And, if I could work, at the same time, as an IT professional and as a teacher, that would be perfect!








[1] Man for Himself, an Inquiry into the Psychology of Ethics;  Escape From Freedom (see  (http://www.amazon.com/Man-Himself-Inquiry-Psychology-Ethics-ebook/dp/B00BPJOD8K/ref=asap_B000APK2LW?ie=UTF8 and http://www.amazon.com/Escape-Freedom-Erich-Fromm-ebook/dp/B00BPJOC7W/ref=asap_B000APK2LW?ie=UTF8 ).
[2] When we leave the terrain of the simpler algorithms to enter into heuristics, the programmer can’t preview his creature’s behavior; I think this is one of the aspects of “creativity”. 
[3] Because religion is a matter of faith and due to this intrinsically un-explainable and un-understandable.
[4] Yes, I’m completely aware of how ridiculous this can sound.  I’m also completely aware that I am the naïve kind of man who is deeply moved by heroic melodramas.

Wednesday, December 10, 2014

CIS562 - Enterprise Information Systems - Airline Flights Finder Project

              

 KANSAS STATE UNIVERSITY

CIS562 - ENTERPRISE INFORMATION SYSTEMS

Fall 2014 Semester

PROJECT: Airline Flights Finder Application

Student: Pedro Francisco Borges Pereira
Under the guidance of
Professor Stephane Faroult

Context
The MS-Access application we describe in this document is intended to be a prototype.
It shall be used as a guideline, a specification, to build a Web Application, using .Net, PHP, or other equivalent tool.
It is an “academic project”, it is not sophisticated enough to a “real life” use, but, still, we think that everything that has been implemented makes sense.  Maybe you can even find some interesting logical approaches and useful programming techniques.
Most of the application business logic has been built on the database layer - SELECT commands, views, User Defined functions.  It can be easily converted to other databases - SQL Server, Oracle, MySQL.
There is some Visual Basic for Applications code to "fire" the database engines.  We think it is easily implementable in Java, C#, PHP and other languages.

Overview
This project’s goal is to build an application to find Flights between “City A” and “City B”. 

Has been developed as a homework in the course CIS 562 – Enterprise Information Systems, under guidance of Professor Stephane Faroult, at Kansas State University (Manhattan, Kansas) in the Fall 2014 semester.

If you want to see demonstrations of how to query a relational database, I'm sure you'll find this pretty interesting, with examples of simple SELECTs, JOINS, SELECT from views, aggregate functions, (reasonably) sophisticated filter criteria, and cartesian products.
User Defined Functions in SELECT lists - to calculate the distance between two Airports using Latitude and Longitude, and to calculate the Arrival Time taking in account the Timezones - are an useful feature, also.
As a by-product, the app also contains a recursive algorithm.  That's how the app can find any number of needed connections.

To read more, download the PDF Document.
To run this application, download the MS-Access file.



Monday, December 8, 2014

Cybersecurity

Cybersecurity

If I was in charge of protect my company’s computer systems, I think I’d have to balance “accessibility” with “security”.
Continuity of operations depends on the computer systems and the network, but protect them shall not hinder business processes.  We can’t - as an extreme example - just decide not using the Web.  This would be safe, for sure, but a company can’t work this way.  And, since you are in the Web, you are at risk.  The question is to take only the risks that worth, and manage it.

The network can’t stop.  File servers and web servers must be available, and its content’s integrity must be granted.
We must always keep in mind that all customer data, and the company’s intellectual data, which together composes one of the most valuable assets, are stored into and are accessed thru this IT infrastructure.  This is vital information, this is sensitive information, and from its integrity and confidentiality depends our company’s survival!

I would start by hiring specialized personnel to manage my IT infrastructure.  These resources should be both internal – my own staff – and external – a specialized consultancy.   
I want internal resources because they’ll be aware of my business peculiarities, my critical points, and so on – and probably more strongly motivated to run the extra mile to protect the company than an external consultancy would be. 
An external consultancy, by contrast, will bring to the table problems, kinds of attacks, which they faced with other customers.  Also, they’re constantly searching and assessing new security methodologies and technologies.
My IT Security Manager (or Network Admin, or whatever you’d like to call the “Infrastructure Security Guy”) would be in charge of building a Security Policy, implement it with the best suited technology, and assess and review it continuously.  The external consultancy would support all these activities.

Of course, the best security police is worthless if the regular user is not committed to not compromising security.   If a user writes his password and glues it in his monitor, no security policy can work.
I would ask my IT Security Manager to define, technically, a Security Guide for all users.   
Then I would ask my Human Resources department to review it and – without changing any technical point – make it as readable and engaging and persuasive as possible.  Being aware of this security procedures and policies would be mandatory to all new employees.

I would also use my external security consultancy to act as a “mystery shopping consumer”, i.e., trying to “fish” sensitive information, anonymously, either using technologies and/or “real world”, “physical”, approaches.  This would help to review and enhance our security policies and procedures.

Last, but not least, despite the fact that in Brazil most big companies are using contractors for cleaning and as security guards, I’d not.  I want all the personnel with access to company buildings – mostly out of business hours – being actually part of the company, part of the team.  I want them hired, trained and audited by the company itself.  I want them knowing the people they work with, and being known by them.  Contractors would be a security fail, in my point of view.








Monday, December 1, 2014

ROBOTS IN THE FUTURE

ROBOTS IN THE FUTURE

What is a Robot?


To write this article, first thing I did was searching for “robot” definitions.

Asking Google I got the following definition:
a machine capable of carrying out a complex series of actions automatically, especially one programmable by a computer.
synonyms:  automaton, android, golem; (especially in science fiction) a machine resembling a human being and able to replicate certain human movements and functions automatically.
synonyms: automaton, android, golem; used to refer to a person who behaves in a mechanical or unemotional manner - "public servants are not expected to be mindless robots"

I asked Wikpedia also:
A robot is an automatic mechanical device often resembling a human or animal. Modern robots are usually an electro-mechanical machine guided by a computer program or electronic circuitry. Robots can be autonomous or semi-autonomous and range from humanoids such as Honda's Advanced Step in Innovative Mobility (ASIMO) and TOSY's TOSY Ping Pong Playing Robot (TOPIO) to industrial robots, collectively programmed swarm robots, and even microscopic nano robots. By mimicking a lifelike appearance or automating movements, a robot may convey a sense of intelligence or thought of its own.

And the Merrian-Webster Dictionary:
a real or imaginary machine that is controlled by a computer and is often made to look like a human or animal
a machine that can do the work of a person and that works automatically or is controlled by a computer


Will Robots be Used in Almost all Human Activities?

Yes, indubitably, yes! 
Nowadays, robots are already used in any area where “a machine capable of carrying out a complex series of actions automatically, especially one programmable by a computer” can replace human work.
The robots that we already have are great to
·         Execute tasks that require strength and/or precision and consist mostly in repeated actions, like for example in auto plants – actually in most big plants, in any area. 
·         Work in dangerous situations, as for example firefighting, submarine equipment maintenance, working in underground pipes, and so on.

Robots are increasing being used to control airplanes.   They can use the information provided by sensors like wind speed meters, gyroscopes, GPS’s and so on to pilot the aircraft.
But human pilots are indispensable, their presence is mandatory.  But not all pilots – and not all passengers – agree in taking from the pilot the option of taking control whenever he thinks he should.
I think that shows that people are amazed by what robots are capable to do, but are not – still – wanting to trust their lives in a robot’s “hands”.
Interestingly, the same objections are not raised when it comes to control missiles.  Apparently, when a robot’s malfunctioning can kill “other people” – enemies, or just strangers, trusting a robot is acceptable.

I think is just a matter of time to people forget about their objections to putting their lives in robot’s “hands”.

Let me share an experience:  I’m Brazilian, in Brazil we do not have a lot of automatic cars, and I never drove a car with an “automatic pilot” before I arrived at United States.  Don’t think we can call the automatic transmission a kind of robot?  It is in charge of changing the engine’s gear, a task that we Brazilians have to repeat like “a zillion” times in our daily lives.  When you’re in a traffic jam, this means a lot of work!  And, eventually, am not I trusting my life to this robot?  I mean, what if I try to speed the car to get out of the way of another vehicle, and it does not work?
And I have to tell that when I fell the car “taking me” the throttle when I turn on the “automatic pilot”, I feel scared!  It’s clear to me that I’m not controlling the car anymore! I can’t help but thinking “what if it does not returns the control to me when I need?”
But I don’t believe that American drivers feel that way.
So, it’s just a matter of time, of getting used to trust in robots, and our fear will not hinder their spread anymore.

Antropomorphic vs Non-Antropomorphic Robots

I do believe that we’ll have a very meaningful presence of robots in a near future.  But I think that most of them will not have human appearance. I think they will be “autonomous machines”.   
We already have automatic plows and harvesters.  A human being needs to be present, but I think that in a few years it will not be required anymore.
Cars that will transport us, being enough that we provide the destination – similarly to what we do with GPS devices nowadays, will be something standard;  an human driving a car will be an eccentricity.

All kinds of machines will have this “embedded” robots, dispensing human operators.
I think that this makes more sense than building anthropomorphic robots to control the machines. 

Domestic Robots

All housekeeping will be done by robots.  Not just dishwashing, laundry and cleaning.  I think that we’ll be able to define a “kind” of diet we’d like to follow, and a “robot mother” will take care of all details, buying all groceries – this in case we still cook at home in a few years – and/or pre-cooked food, and serving us the meals.
Maybe these domestic robots will be designed to have human appearance, but I think that it can be more virtual than physical.  Something like a hologram to interact with us, while the housework tasks are executed by robots built similarly to our current appliances.

Activities beyond Robot’s Range

All activities which requires human empathy cannot be delegated to robots.  They can “mimic” human empathy, but they are not humans and due to this a “robotic psychotherapist” cannot be as satisfying as its human counterpart.  And can you imagine a “robot pastor”? 
More worldly human activities shall also be forbidden to robots.  What if instead of having a “robot mother” to take care of your house, you opt for having a “robot wife”?  Of course it would bring you your flip-flops and a beer and be quiet while you watch the football game or listen to you telling about your day, whatever you prefer.  Could you fall in love with a machine?   And could you make… No, I don’t want even think about this, it’s far too scary!  If this thought don’t disgusts you, I suggest you talk to a psychotherapist, or a pastor - human ones, please!

Finally I’d like to highlight that despite robots are great taking care of elder people, these robots should not have human appearance.  We shall not forget that they are just “improved canes”.  They cannot and they shall not replace the care and the affection of children and grandchildren.


How will This World look alike?

Honestly, I don’t know, I don’t think we can preview.  Isaac Asimov, for example, had some genial insights – I love his “theory of chaos applied to human societies” – but his descriptions of future societies still using pencil and paper, for example, were wrong.  George Orwell, with his “Big Brother”, also describes in a way amazingly plausible a sick society, but in terms of technology, he was also wrong.  Arthur Clarke, with his “2001”, describes something similar to “The Singularity” – but, again, the timeline is completely wrong.
So I admit that if these genial writers were not able to preview the future of technology, I certainly am not capable.
One thing I know for sure: it will amaze all of us who come to see it!