Spark for Python Developers by Amit Nandi

By Amit Nandi

Key Features

  • Set up real-time streaming and batch info extensive infrastructure utilizing Spark and Python
  • Deliver insightful visualizations in an internet app utilizing Spark (PySpark)
  • Inject reside info utilizing Spark Streaming with real-time events

Book Description

Looking for a cluster computing approach that gives high-level APIs? Apache Spark is your answer—an open resource, speedy, and normal goal cluster computing procedure. Spark's multi-stage reminiscence primitives supply functionality as much as a hundred occasions speedier than Hadoop, and it's also well-suited for computer studying algorithms.

Are you a Python developer susceptible to paintings with Spark engine? if this is the case, this publication can be your spouse as you create data-intensive app utilizing Spark as a processing engine, Python visualization libraries, and internet frameworks reminiscent of Flask.

To commence with, you are going to study the best solution to set up the Python improvement atmosphere powered through Spark, Blaze, and Bookeh. you are going to then tips to connect to info shops corresponding to MySQL, MongoDB, Cassandra, and Hadoop.

You'll extend your talents all through, getting familiarized with a number of the information resources (Github, Twitter, Meetup, and Blogs), their information constructions, and options to successfully take on complexities. you will discover datasets utilizing iPython pc and may detect the best way to optimize the information types and pipeline. eventually, you will get to grasp the right way to create education datasets and teach the computing device studying models.

By the tip of the booklet, you've created a real-time and insightful pattern tracker data-intensive app with Spark.

What you are going to learn

  • Create a Python improvement atmosphere powered by way of Spark (PySpark), Blaze, and Bookeh
  • Build a real-time pattern tracker information in depth app
  • Visualize the developments and insights won from facts utilizing Bookeh
  • Generate insights from facts utilizing computer studying via Spark MLLIB
  • Juggle with information utilizing Blaze
  • Create education info units and educate the computer studying models
  • Test the desktop studying types on try out datasets
  • Deploy the computer studying algorithms and types and scale it for real-time events

About the Author

Amit Nandi studied physics on the loose college of Brussels in Belgium, the place he did his examine on computing device generated holograms. machine generated holograms are the main elements of an optical computing device, that's powered through photons working on the velocity of sunshine. He then labored with the college Cray supercomputer, sending batch jobs of courses written in Fortran. This gave him a style for computing, which saved starting to be. He has labored largely on huge company reengineering projects, utilizing SAP because the major enabler. He targeted for the final 15 years on start-ups within the facts house, pioneering new components of the knowledge expertise panorama. he's at present targeting large-scale data-intensive functions as an firm architect, information engineer, and software program developer. He is aware and speaks seven human languages. even if Python is his laptop language of selection, he goals on the way to write fluently in seven machine languages too.

Table of Contents

  1. Setting Up a Spark digital Environment
  2. Building Batch and Streaming Apps with Spark
  3. Juggling info with Spark
  4. Learning from info utilizing Spark
  5. Streaming reside facts with Spark
  6. Visualizing Insights and Trends

Show description

Read Online or Download Spark for Python Developers PDF

Best other_3 books

Shaw's Textbook of Gynecology E-Book

Shaw's Textbook of Gynecology publication

The Rough Guide to South America On a Budget (Rough Guide to...)

The full-colour The tough advisor to South the US on the cheap is the final word guidebook for traveling the continent, from Colombia's Caribbean shores to the ice-fields of Patagonia. special color maps and in-depth insurance of what to work out, the place to stick and the way to get round may help you find the easiest this outstanding continent has to supply.

Sacred Heart

The kids of U. S. small-town Alexandria are only attempting to dwell like basic adolescents till their mom and dad' promised go back from a mysterious, four-year non secular pilgrimage, and Ben Schiller isn't any exception. She's simply attempting to look after her sister, hold religion that her mom and dad will come again, and get via her teenager years as painlessly as attainable.


Like many outgoing younger women, Fatima feels rebellious opposed to mom and dad she sees as strict. It in order that occurs that she is Egyptian-born and wears a hijab. whilst anti-Muslim graffiti appears to be like at the partitions of her institution, Fatima transfers to a brand new tuition. The information counsellor there, Mr. E. , does his top to aid Fatima slot in, yet regardless of his suggestion she starts off an not going friendship with Jorah, who has a name for anger matters.

Extra info for Spark for Python Developers

Sample text

Download PDF sample

Rated 4.61 of 5 – based on 7 votes