Big Data Engineer

£40,000 – £55,000

Location: United Kingdom (London – City of London, London) Type: Permanent

Big Data Engineer

London

Permanent

Salary: £40,000 - £55,000

Anson McCades currently recruiting a Big Data Engineer for our Global Security Solutions client based in London. Our client has over 89,000 people in more than 40 countries. They develop, engineer, manufacture and support systems and products to protect national security and people, deliver military capability and keep critical information and infrastructure secure. The Big Data Engineer will join existing project teams working directly with for Central Government customers.

Big Data Engineer duties will include:

• To design, implement and test scalable and performant analytic pipelines - with guidance from a Data Scientist and Solution Architect.
• To contribute to the continuous improvement of the engineering processes and architecture for the analytics solution.
• To detect and analyse performance bottlenecks in new and existing solutions.
• To tune analytic implementations to cope with the solution scale - we ingest millions of events per second.
• To communicate and act on solution improvement ideas.
• To rapidly learn new technologies.

The ideal Big Data Engineer will have experience in:

• Knowledge of the following is essential
• Python, our code base is Python based
• Knowledge of the following big data technologies would be extremely advantageous
• Spark / PySpark (all data analysis on a Spark cluster)
• HDFS (forms our persistence backbone)
• HBase / Elasticsearch databases (used to store entity data)
• Kafka(for messaging / subsystem buffering)Knowledge of the following would be desirable but not essential
• Ansible (our solutions are installed using Ansible)
• Airflow/ Jupyter (for job control and one off tasks
• A JVM language e.g. Java / Scala
• Unix shell scripting (BASH for example)·Data formats such as Parquet and Avro

In return you will receive:

• Competitive salary
• Hybrid Working
• Permanent employment

If you are interested in applying for the Big Data Engineer role, simply:

• Click apply, call Peter Donaldson on 02077806706 or email your CV

Reference: AMD/PD/BDE/001