Sunday, April 23, 2017

Why we need Big Data

The process of storing and analysis data to make sense for the organization is called Big data. In simple terms,data which is very large in size and yet growing exponentially with time is called as Big data.The volume of data being made publicly available increases every year, too. Organizations no longer have to merely manage their own data; success in the future will be dictated to a large extent by their ability to extract value from other organizations’ data.

For any application that contains limited amount of data we normally use SQL/Oracle/ MySQL,but what in case of large applications like Facebook,Google,YouTube? This data is so large and complex that none of the traditional data management system is able to store and process it.

Facebook generates 500+ TB data per day as people upload various images,videos,posts etc..Similarly sending text/multimedia messages,updating Facebook/whatsapp status,comments etc..generates huge data.If we use traditional data processing applications(SQL/ORACLE/MySQL)to handle it, it will lead to loss of efficiency. So in order to handle exponential growth of data,data analysis becomes a required task. To overcome this problem,we use Big data. Big data includes both structured and unstructured data.

Structured Data means the data which can be stored and processed in table format is called as a structured data. it is very simple to enter, store and analyze.Example: RDBMS

Unstructured Data means the data with unknown form or structure is called as unstructured data. Example: Text files,images,videos,webpages,PDF files,PPT,social media data etc..

Semi structured Data means combination of both Structured and unstructured data.Example: XML data. 


Traditional management systems and existing tools are facing difficulties to process such a big data.R is one of the main computing tool used in statistical education Research. It is also widely used for data analytics and numerical computing in scientific research.

This type of Big Data come from Social Media,E-Commerce,Share Market and Airplane etc..

1.Social Media: This could be data coming from social media services such as Face Book Likes,photos and videos uploads,putting comments,Tweets and You Tube views.Facebook hosts more than 240 billion photos, growing at 7 petabytes per month.
2. Transport Data: Transport Data includes model,capacity,distance and availability of a vehicle.
3. Share Market:  Stock Exchange generates huge amount of data through its daily transactions.The New York Stock Exchange generates about 4−5 terabytes of data per day.

4. E-Commerce site: E-Commerce Sites Like FlipKart,Amazon,Snapdeal generates huge amount of data.
5.Search Engine Data: Search engines retrieve lots of data from different databases.


6.Airplane: single Airplane can generate 10+ TB of data in 30 Minutes of flight time.

Then you may raise question what is the need to store huge amount of data. Then here is answer.The main reason behind storing data is analysis.Data analysis is a process used to clean,transform and remodel data with a view to reach to a certain conclusion for a given situation.More accurate analysis leads to better decision making leads to increase in efficiency and risk reduction.

Example: When we search on e-commerce websites(FlipKart,Amazon)we get some recommendations of product that we search. The analysis of data that we entered is done by these websites,then accordingly the related products are displayed.

Example: when we search any smart phone,we get recommendations to buy back covers,screen guard etc..

Similarly,Facebook stores our images,videos? The reason is advertisement. 
There are two types of marketing. They are
a)Global Marketing: show advertisement to all users.
b)Target Marketing: Show advertisement to particular groups/people. So in target marketing,Facebook analysis it's data and it shows advertisement to selected people.

Example: If advertiser wants to advertise for cricket kit and he/she wants to show that advertisement to only interested set of people. So Facebook tracks a record of all those people who are member of cricket groups or post anything related to cricket and displays it to them.

Watch Video: What is Big Data?

ALSO READ:

Introduction to Hadoop
What are the prerequisites to learn Hadoop
Basic Hadoop Interview Questions


Saturday, April 22, 2017

Java Vector Class with Examples

A Vector class also stores elements similar to ArrayList,but Vector is Synchronized.Vector class implements the List interface.The underlying Data Structure is re-sizable array or growable array. Every method present in the Vector class is synchronized and vector object is Thread-safe.

Creating Vector Class:

1) Vector v=new Vector();//creates an empty vector object with default initital capacity 10
                                                       (OR)
2) Vector v=new Vector(int initialCapacity);//creates a vector 'object' with the specified capacity.Whenever an element is added to the initial capacity,the capacity of the vector is doubled.
 Like this:  new capacity=currentcapcity*2
                                                        (OR)
3) Vector v=new Vector(int initialCapacity,int incrementCapacity)

Vector specific methods:

boolean add(E e)   -------->  Appends the element e to the end of the list

void add(int i,E element)--------> inserts the element e at the specified index 'i'in the list

void addElement(Object 0)------>inserts the element to the vector and increses the vector size by 1,This method is synchronized.

To REMOVE objects:

element remove(int index)  -----> This method removes the element at the specified position in the vector.This method also returns the element which was removed from the Vector.

boolean remove(Object o)  ------> This method removes the first occurrence of the specified element object from the vector,if it is present

void clear() -----> This method removes all the elements from the vector.

To Access Elements:

Object get(int index) -----> This method returns the element available at the specified position in the vector

int size() -----> This method returns the number of elements present in the Vector.

Example:
import java.util.*;
class VectorDemo
{
public static void main(String arg[])
{
Vector v = new Vector();
System.out.println(v.capacity());
for (int i = 0;i<10 ;i++ )
{
v.addElement(i);
}
System.out.println(v.capacity());
v.addElement("Aa");
System.out.println(v.capacity());
System.out.println(v);
}
}
Output:







Example 2:

This example shows creating a vector with Integer elements

import java.util.*;
class VectorTest
{
public static void main(String args[])
{
Vector<Integer>v=new Vector<Integer>();
//take an int type array
int x[]={12,23,34,45,55};
for(int i=0;i<x.length;i++)
{
v.add(x[i]);
}
//retrieve the elements using get()
System.out.println("vector elements");
for(int i=0;i<v.size();i++)
{
System.out.println(v.get(i));
}
//retrieve using ListIterator
System.out.println("Elements using ListIterator");
ListIterator li=v.listIterator();
System.out.println("In forward direction:");
while(li.hasNext())
System.out.println(li.next()+"\t");
System.out.println("Backward Direction:");
while(li.hasPrevious())
System.out.println(li.previous()+"\t");
}
}
Output:











ALSO READ:
How to prepare for campus interviews
Java for each loop with example
How Java PROGRAM WORKS
Java Object class methods with examples

                                                      

Friday, April 21, 2017

How to Solve Producer Consumer Problem in Java

Producer Consumer Problem is an important design concept involving multi-threading and inter thread communication. The main concept is that,Producer thread will produce the items to the Queue Consumer thread will Consume items from the Queue.If the Queue is empty Consumer thread has to call wait() method on the Queue object then it will enter into waiting state.After producing the items Producer thread has to give notification so that Consumer thread will get that notification and consume items.

Pseudo code to solve produce consumer problem:


Example:
This program shows how to solve producer consumer problem using wait() and notify()

class InterThread
{
public static void main(String args[])throws Exception
{
//producer produces some data which consumer consumes
Producer p1=new Producer();
//pass producer object to consumer so that it is then available to consumer
Consumer c=new Consumer(p1);
//create two threads and attach producer and consumer
Thread t1=new Thread(p1);
Thread t2=new Thread(c);
//Run the threads
t2.start();//consumer waits
t1.start();//producer starts production
}
}
class Producer extends Thread
{
//to add data,we use string buffer object
StringBuffer sb;
Producer()
{
sb=new StringBuffer();//allot memory
}
public void run()
{
synchronized(sb)
{
//go on appending data to string buffer
for(int i=1;i<=10;i++)
{
try{
sb.append(i+":");
Thread.sleep(2000);
System.out.println("adding");
}
catch(Exception e)
{}
}
//data production is over,so notify to consumer thread
sb.notify();
}
}
}
class Consumer extends Thread
{
//creates producer refercence to refer to producer object from consumer class
Producer prod;
Consumer(Producer prod)
{
this.prod=prod;
}
public void run()
{
synchronized(prod.sb)
{
//wait till a notification is received from producer thread
try{
prod.sb.wait();
}
catch(Exception e)
{}
//when data production is over,display data of string buffer
System.out.println(prod.sb);
}
}
}

Output:
adding
adding
adding
adding
adding
adding
adding
adding
adding
adding
1:2:3:4:5:6:7:8:9:10:

Read Also:

InterThread Communication
Synchronization Interview Questions
How to find duplicate elements in array



Saturday, April 15, 2017

Java Recursion with example

A method is calling itself is known as recursive method.Java programming language allows a method to call itself this is nothing but 'recursion'. Many Programming Problems solved by the recursion.The main advantage to recursive methods is that they can be used to create clearer and simpler versions of simpler algorithms that can their iterative relatives.For example, A Quick Sorting algorithm is very difficult to implement in iterative way. For Most of the algorithms implements in easy manner using recursion. 

Recursive versions of many routines may execute a bit slow compare to iterative process. Since when a method calls itself, new local variables and parameters are allocated storage on the stack, and the method code is executed with these new variables from the start. As each recursive call returns, the old local variables and parameters are removed from the stack, and execution resumes at the point of the call inside the method.Because storage for parameters and local variables is on the stack and each new call creates a new copy of these variables, it is possible that the stack could be exhausted. If this occurs, the Java run-time system will cause an exception says StackOverFlowException.

Factorial program using for loop(iterative method):
class factorial
{
//recursive declaration of method factorial
public long factorial(long num)
{
long result=1;
//iterative declaration of method factorial
for(long i=num;i>=1;i--)
result=result*i;
return result;
}
public void displayfact()
{
//calculate the factorials 0 through 5
for(int counter=0;counter<=5;counter++)
System.out.printf("%d!=%d\n",counter,factorial(counter));
}
}
public class FactorialTest
{
public static void main(String args[])
{
factorial f=new factorial();
f.displayfact();
}
}
Output:
0!=1
1!=1
2!=4
3!=6
4!=24
5!=120


Let us understand this concept with factorial example using recursion:

class RecursionDemo
{
static long fact(int num)

{
long result;
if(num==1)
return 1;
result=fact(num-1)*num;
return result;
}
public static void main(String args[])
{
System.out.println("factorial of 5:");
System.out.println(RecursionDemo.fact(5));
}
}

Output:
factorial of 5:
120



Friday, April 14, 2017

DAO Design Pattern

The Data Access Object(DAO) is one of the J2EE pattern. The DAO pattern hides the implementation details of data source from it's clients,so introducing loose coupling between your core business logic and your persistence mechanism. The main purpose of this Design pattern is to implement persistence layer of the Java application.Here persistence store means all database software like oracle,MySQL.

Let us discuss now without DAO in real projects:

In MVC architecture projects or other projects if persistence logic is mixed up with other logic of the application especially (Business logic)then persistence logic  will not become flexible to modify. That means the modification done in persistence logic may disturb other logics of the application.So, DAO pattern main purpose is to provide a clean design-oriented mechanism for accessing all types of data from your application.

Solution: 
The solution is working with DAO design patter. The java class that separates persistence logic from other logic of the application to make persistence logic as flexible logic to modify is called as DAO design pattern.

DAO class contains:

  • It contains the logic to establish the connection
  • It contains the logic to release the connection
  • It contains the logic to perform persistence operation like insert,update,delete and select
All the above logics of DAO can be developed by using any persistence technology like JDBC,Hibernate,entity Bean components etc...

All the Business components of project can use either single DAO or they can use separate DAO class for each Business components .It is always recommended to write separate DAO class for each Business component.

Now we will understand DAO design pattern with Employee details example:

Create Employee table in MySQL:

CREATE TABLE Employee ( 
EmpId int(1)  PRIMARY KEY NOT NULL, 
Name varchar2(20) NOT NULL, 
DeptName varchar(25) NOT NULL, 
Address text 

 );

Now we will write a java program to make DBConnection:

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
public class DBConnector {
String driverClassName = "com.mysql.jdbc.Driver";
String connectionUrl = "jdbc:mysql://localhost:3306/Employee";
String dbUser = "root";
String dbPwd = "root";
private static DBConnector DBConnector = null;
private DBConnector() {
try {
Class.forName(driverClassName);

catch (ClassNotFoundException e) 
{
e.printStackTrace();
}
}
public Connection getConnection() throws SQLException {
Connection conn = null;
conn = DriverManager.getConnection(connectionUrl, dbUser, dbPwd);
return conn;
}
public static DBConnector getInstance() {
if (DBConnector == null)
{
DBConnector = new DBConnector();
}
return DBConnector;
}
}

Now we will write code for class EmployeeBean:


import java.io.Serializable;
public class EmployeeBean implements Serializable
{
int EmpId;
String name;
String DeptName;
String address;
public EmployeeBean() 
{
}
public StudentBean(int EID, String name, String DeptName, String address) 
{
this.EmpId = EID;
this.name = name;
this.DeptName = DeptName;
this.address = address;
}
public int getEmpId() 
{
return EmpId;
}
public void setEmpId(int EmpId) 
{
this.EmpId = EmpId;
}
public String getName() 
{
return name;
}
public void setName(String name) 
{
this.name = name;
}
public String getDeptName() 
{
return DeptName;
}
public void setDeptName(String DeptName) 
{
this.DeptName = DeptName;
}
public String getAddress() 
{
return address;
}
public void setAddress(String address) 
{
this.address = address;
}
}

Now we will write EmployeeDAO class:

import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
public class EmployeeDAO 
{
Connection connection = null;
PreparedStatement ptmt = null;
ResultSet resultSet = null;
public EmployeeDAO() 
{
}
private Connection getConnection() throws SQLException 
{
Connection conn;
conn = DBConnector.getInstance().getConnection();
return conn;
}
public void add(EmployeeBean EmployeeBean) 
{
try {
String queryString = "INSERT INTO Employee(EmpId, Name, DeptName, Address) VALUES(?,?,?,?)";
connection = getConnection();
ptmt = connection.prepareStatement(queryString);
ptmt.setInt(1, EmployeeBean.getEmpId());
ptmt.setString(2, EmployeeBean.getName());
ptmt.setString(3, EmployeeBean.getDeptName());
ptmt.setString(4, EmployeeBean.getAddress());
ptmt.executeUpdate();
System.out.println("Data Added Successfully");
}
catch (SQLException e) 
{
e.printStackTrace();
finally 
{
try {
if (ptmt != null)
ptmt.close();
if (connection != null)
connection.close();
} catch (SQLException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
}
public void update(EmployeeBean EmployeeBean) {
try {
String queryString = "UPDATE Employee SET Name=? WHERE EmpId=?";
connection = getConnection();
ptmt = connection.prepareStatement(queryString);
ptmt.setString(1, EmployeeBean.getName());
ptmt.setInt(2, EmployeeBean.getEmpId());
ptmt.executeUpdate();
System.out.println("Table Updated Successfully");
} catch (SQLException e) {
e.printStackTrace();
} finally {
try {
if (ptmt != null)
ptmt.close();
if (connection != null)
connection.close();
}
catch (SQLException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
}
public void delete(int Empid) {
try {
String queryString = "DELETE FROM Employee WHERE EmpId=?";
connection = getConnection();
ptmt = connection.prepareStatement(queryString);
ptmt.setInt(1, EmpId);
ptmt.executeUpdate();
System.out.println("Data deleted Successfully");
} catch (SQLException e) {
e.printStackTrace();
} finally {
try {
if (ptmt != null)
ptmt.close();
if (connection != null)
connection.close();
} catch (SQLException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
}
public void findAll() {
try {
String queryString = "SELECT * FROM Employee";
connection = getConnection();
ptmt = connection.prepareStatement(queryString);
resultSet = ptmt.executeQuery();
while (resultSet.next()) {
System.out.println("EmpId " + resultSet.getInt("EmpId")
+ ", Name " + resultSet.getString("Name") + ", DeptName "
+ resultSet.getString("DeptName") + ", Address "
+ resultSet.getString("Address"));
}
catch (SQLException e) {
e.printStackTrace();
} finally {
try {
if (resultSet != null)
resultSet.close();
if (ptmt != null)
ptmt.close();
if (connection != null)
connection.close();
catch (SQLException e) 
{
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}

Main class:

public class MainClass {
public static void main(String[] args) {
EmployeeDAO employee = new EmployeeDAO();
EmployeeBean emp = new EmployeeBean();
emp.setName("lucky");
emp.setEmpId(6);
emp.setDeptName("Techinical");
emp.setAddress("hyderabad");
EmployeeBean eb = new EmployeeBean();
eb.setName("mahesh");
eb.setEmpId(5);
// Adding Data
employee.add(emp);
// Deleting Data
employee.delete(4);
// Updating Data
employee.update(eb);
// Displaying Data
employee.findAll();
}
}

ALSO READ:

JDBC INTERVIEW QUESTIONS
SPRING INTERVIEW QUESTIONS
STRUTS INTERVIEW QUESTIONS
TOP 20 HR INTERVIEW QUESTIONS
HIBERNATE INTERVIEW QUESTIONS






Tuesday, April 11, 2017

Project Release Process to Client Organisation

It is the main phase of SDLC which talks about releasing project to client organization from software company after its completion in development phase. The organization who give their requirements to software company for developing software projects is called as client organization. For example,SBI,ICICI,LIC etc...The company who develops the projects for client organization is called as software company.Example: IBM,TCS,HP etc...Once the project is released then the client organization location is called onsite location and software company location is called as offshore location. 

The Integrated Machine is the central Machine of Software Development company where all projects of company will be integrated,tested and managed. This machine generally runs on Linux or solaris Operating System.While releasing project certain members of team will be asked to go to client organization location to receive the project and to install the project. This team is called as onsite team. The remaining team members will stay in software company itself to support onsite team. This supporting team is called offshore team. The machine of client organization where the received project will be installed is called as production machine or box.



Step by Step project release process:


  • Team completes Integration of project
  • In house testing takes place on the project
  • 3rd party Testing takes place on the project
  • Project Manager forms onsite team and sends to client organization and also forms offshore team to support onsite team
  • offshore team prepares tar file(taped archive) like zip file of windows having all the resources of projects like(.java, .class, .properties, .jar  , .xml)
  • offshore team prepares Dispatch Central(D.C) containing instructions of receiving and installing projects.
  • onsite engineers keeps production box ready having the same setup of integrated machine(including software machine).
  • offshore team uses FTP application and releases tar file project to client organization having D.C document
  • Project Leader keeps release mail to various people like PM,HR department,delivery manager,client organization representative,onsite and offshore team members.
  • onsite engineers receive the project and install the projects in production box by following D.C document instructions.
  • onsite team creates some dummy user in the project makes the employees of the client organization to test project for few days.This is called User Access Test(UAT).
  • If UAT results are positive then client organization sends congrats mail to offshore team/software company. That means end of project release.
All the above steps indicate main release of the project.In maintenance phase of the project the client organization sends bugs and issues to offshore team through onsite team. This offshore team will improvise the code or will write new code to fix bugs and issue and they supply this code to onsite team.This onsite team will replace old code with new code based on new D.C.document instructions. This process is called as patch releases of project.

Watch Video For More Details: Project Release Process to Client Organization




Saturday, April 8, 2017

Java Daemon Thread with Example

The Threads which are executing in the background are called Daemon Threads. A thread has to continuously execute without any interruption to provide services to other threads or objects.It generally provides a background processing. The main objective of Daemon Thread is to provide support for non-Daemon threads. For Example: Garbage Collector.

We can check whether the Thread is Damon or not by using public final boolean isDaemon() method. We can change Daemon() nature of a thread by using setDaemon() method.
public final void setDaemon(boolean b)

We must remember,we can change Daemon() nature before starting thread only. If after starting a thread if we are trying to change demonstration then we will get Run time Exception says illegalThreadstateException.

main() thread is always non-daemon() and we can not change it's Daemon() nature because it is already started at the beginning only.If the parent class is Daemon() then child class is also Daemon() and if the parent is non-Daemon() , child is also non-daemon()

Note: whenever last non-daemon() thread terminates automatically all Daemon() threads will be terminated.

Example:
class myThread extends Thread
{
public void run()
{
for(int i=0;i<10;i++)
{
System.out.println("Lazy Thread");
try{
Thread.sleep(2000);
}
catch(InterruptedException ie)
{
}
}
}
}
class DaemonThreadDemo
{
public static void main(String args[])
{
myThread t=new myThread();
t.setDaemon(true);         
t.start();
System.out.println("end of main method");
}
}
Output:
End of main method
Lazy Thread

If we are comment t.setDaemon(true) then both main() and child() threads are non-Daemon() and hence both will be executed until their completion. If we are not comment t.setDaemon(true) then main() method is non-Daemon() and child thread is Daemon() and hence whenever main() thread terminates automatically child thread will be terminated.

ALSO READ TOP POSTS:
Difference between web server and application server
Common interview Questions and answers
Java Exception interview Questions 
What are the First things you do after joining the company
inter thread communication in java with examples
When to use abstract class and interface in Java

Java Memory Management Interview Questions

In this post i am sharing memory management in java interview questions. These are frequently asked interview questions for java developers. Let us read what are those?

1) What is memory management in java?

Ans: Memory management is the process of allocating new objects and removing unused objects to make space for those new object allocations. However, the java developer does not need to explicitly allocate and deallocate memory. Since JVM has responsibility to handle management in java.

2)What are the different segments of memory?

Ans: There are 3 types of segments of memory. They are

  1. Stack
  2. Heap
  3. Code
stack segment: It contains local variables and reference variables
heap segment: It contains all created objects at runtime and instance variables
Code segment: This segment loads the java code that means where actual compiled java byte codes resides when loaded.

3) What is Garbage Collection?Advantages of it?

Ans: In ‘C++’ the programmer is responsible for both creation and destruction of objects but usually the programmer is giving very much importance for creation of objects and he is ignoring the destruction of objects.But in java programmer is responsible only for creation of objects but sun people has introduced one assistance which is running continuously in the background for destruction of objects. Due to this assistance there is no chance of failing the java program due to memory problem, this assistance is nothing but "Garbage Collector".

 The main advantage of Garbage Collection is it removes the burden of manual memory allocation/deallocation from us,so that we can focus on solving the problem

4) How Garbage Collector Works?

Ans: The Java runtime environment delete objects when it determines that they are no longer being used. This is nothing but Garbage collection.The JVM uses the Mark and Sweep Garbage collection model for performing Garbage collection of the whole heap.
                These consists of two phases they are mark phase and sweep phase. During the mark  phase all objects that are reachable from java threads,native handlers and other resources are marked as alive. This process identifies and marks all objects that are still used and the rest can be considered Garbage. During sweep phase the heap is traversed to find the gaps between the live objects. These gaps are recorded in free list and are made available for new object allocation.

5) What is class loader?different types of class loaders used by JVM?

Ans: Class loader is part of JVM which is used  to load classes and interfaces. There are 3 types of Class loaders used by JVM. They are

  • Bootstrap
  • Extension
  • System
Bootstrap class loader loads JDK internal classes,java*packages
Extension class loader loads jar files from JDK extensions directory
System class loader loads classes from system classpath

6) What is difference between static Vs Dynamic class loading?

Ans: In static class loading  classes are statically loaded with java's "new" operator whereas dynamic class loading is a technique for programatically invoking the functions of a class loader at run time.

7) What is OutOfMemoryError?

Ans: This Error is thrown that JVM has run out of memory and that the garbage collector is unable to claim any more free memory.

8)How an object becomes eligible for Garbage Collection?
Ans: An object is eligible for garbage collection when no object refers to it, An object also becomes eligible when its reference is set to null. The objects referred by method variables or local variables are eligible for garbage collection when they go out of scope. 

Example:
Integer i = new Integer(7);
i = null; 

8)What is permGen or Permanent Generation?

Ans: The memory pool containing all the reflective data of the Java Virtual Machine itself,such as class and method objects.The permanent generation contains meta data required by the JVM to describe classes and methods used in the application.The permanent generation is populated by the JVM at runtime based on the classes in use by the application. 

9) How subString() method of String class create memory leaks?

Ans: The String class of SubString() method would be create new String object keeping a reference to the whole char array,to avoid copying it. Therefore, you can inadvertently keep a reference to a very big character  array with just a one character String.

10) How Java provides high performance?

Ans: Java uses Just-In-Time compiler(JIT) to provide high performance. It is a program that convert byte code into instructions that can be sent directly to the processor.

Also Read:

What is Java Bean with examples?
MVC Architecture in Java
Hibernate Interview Questions
Spring Interview Questions
Struts Interview Questions
JDBC Interview Questions


High Paying Jobs after Learning Python

Everyone knows Python is one of the most demand Programming Language. It is a computer programming language to build web applications and sc...