Friday, April 26, 2019

Neo4j Spark connector Tutorial

Neo4j having python drivers could be availed from this  location
Neo4j spark connector available at location
It is having drivers for java and scala but not for python

Below are the steps to setup Neo4j- spark scala connector application

1. neo4j spark connector
Connector jar could be downloaded from the location
2. scala
download rpm from location
and install using rpm -i
3. sbt
Download sbt to /opt from location
update /etc/profile with export PATH=$PATH:/opt/sbt/bin

Write scala code or checkout example from location
Update neo4j credentials

Build & Run:
Follow steps in the Readme to build & run the application 

Thursday, May 17, 2018

MIcroservice Case study

Microservices could be implemented in several way. In this case below is the stack used.

  • Django
  • Flask
  • Node.js
  • mysql
  • mongo
  • docker
  • kubernetes
  • ubuntu 

In this usecase API searches keyword "search_term" in the available public repositories and displays top 5 active repositories
There are 3 app services here i.e., Django (DAPP) service, FLASK(FAPP) service, NODE.JS(Nodeapp) service
There are 2 database services Mysql(ms-mysql)service and Mongo(ms-mongo)service.

All five docker images are available at location Docker Hub

Preparing the environment

Install docker and kubernetes as mentioned in the location 

Checkout and start kubernetes files from github location

Starting and Stopping services

Start the kubernetes services/pods/deployments with below steps with the files located in dhub
sh ./
sh ./

For stopping kubernetes services/pods/deployments use below step
sh ./

Consuming APIs

DAPP consumes FAPP, MySql services and exposes API

FAPP consumes Mongo service and exposed API

Nodeapp consumes Mongo service

Friday, February 16, 2018

Deployin django in nginx & uwsgi environment

1. Install uwsgi , nginx 
apt-get install uwsgi, nginx

2. Clone django helloworld program
git clone

3. Copy the helloworld program into /opt folder
cp -r Tutorials/dj_nx_wsgi  /opt

sudo chown -R www-data:www-data /opt/dj_nx_wsgi

4. Create a python virtual environment and install django
cd /opt
virtualenv env
source /opt/env/bin/activate
pip install django

5. Test uwsgi
(env) osboxes@ubuntu:/opt$ uwsgi --http  :8001 --wsgi-file /opt/dj_ng_wsgi/dj_ng_wsgi/  --virtualenv /opt/env --chdir /opt/dj_ng_wsgi/

6. Access from browser

7. Configure nginx
copy nginx.conf from dj_nj_wsgi folder to /etc/nginx/sites-available
make soft link
ln -s /etc/nginx/sites-available/nginx.conf /etc/nginx/sites-enabled/nginx.conf

8.restart nginx
service nginx restart

9. start wsgi
(env) osboxes@ubuntu:/opt$ uwsgi --socket :8001 --wsgi-file /opt/dj_ng_wsgi/dj_ng_wsgi/  --virtualenv /opt/env --chdir /opt/dj_ng_wsgi/

10.from browser access the url.

Friday, March 24, 2017

pylint, python logger

pylint --disable=R,C,W,I,E,F --msg-template='{msg_id}:{line:3d},{column}: {obj}: {msg}' --disable=import-error --rcfile ./to-path/.pylintrc

import logging
import sys

root = logging.getLogger()

fo= open("/Users/sadatravu/Surya/py_scripts/text.log", "w")
ch= logging.StreamHandler(fo)
#ch = logging.StreamHandler(sys.stdout) # this is for logging into stdout

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')


def main():
    for i in range(10):
       logging.debug("message [%d]"%i)


Saturday, October 29, 2016

SPA using Angular

Single Page Application using Angular



  • Angular and knockout comes with data binding
  • Backbone is also quite easy to grasp.
  • Angular is very powerful and deals with complex models

Original blog

Wednesday, October 26, 2016


1. Django project files significance

a. => enables the django project settings path to the environment
b. => this file contains the settings of django project like debug mode, database settings, templates path, static_url path etc.,
c. this file is the controller file which maps the web url to the respective view function/class
d. file => this file will have the environment settings like virtual environment site-packages path, django project path. path of this should be included in the apache configuration file apache2.conf
e. => this file will have the information models and members of the models
f. => this file will have the views functions/classes

2. Django commands

django-admin start-project my_pro => Creates django project
django-admin startapp myapp => creates django application inside the django project
python ./ runserver => runs the development server
python ./ makemigrations => creates the database migration scripts
python ./ migrate => applies the migrations
python ./ inspectdb >  => Creates the models wrt to the already existing database
python ./ collectstatic => collects all the static files in the django project
python ./ dbshell  => Opens the database shell
python ./ shell => opens the django/python shell