# High-performance Python without GIL through type inference and thread-safe memory management in Cython — Nexedi

High-performance Python without GIL through type inference and thread-safe memory management in Cython

Full time Lille
Posted date: Feb 06, 2020

Description

Details

  • Task : High-performance Python without GIL through type inference and thread-safe memory management in Cython
  • Preferred location : Lille (France)
  • Other locations : Paris (France), Munich (Germany)
  • Type : Internship or Job or PhD
  • Function : Developer
  • Duration : 3-6 Months/Permanent
  • Reference : Offer-2020-Cython

Description

Nexedi is looking for a Python and C developer interested in implementing a multi-threaded coroutine and garbage collector extension for the Cython language.

Nexedi has been extending for one year the Cython compiler with experimental features that provide automated memory management, multi-threading without GIL and type inference. The goal of this effort is to improve Python code execution speed with 100x to 1000x acceleration factor.

Cython is a superset of the Python language that introduces basic type inference and static code analysis. It is the language used to develop libraries such as scikit-learn and lxml.

The various efforts of Nexedi - named Cython+ - are already promising. Read for example the blog entry: https://www.nexedi.com/blog/NXD-Document.Blog.Cypclass

The paragraph "An alternative approach to cypclass" describes our short term goals: implement the cypclass idea based on PyObject object protocol rather than on C++ object protocol as it is done currently.

A more long term goal is to implement static analysis of a Python code base and determine subsets of code that can be statically typed and compiled with Cython+, while retaining the flexibility of Python scripting for other parts of the code. This long term goal may be part of full research project based on linear types and type inference systems, based on the concept of progressive types. A PhD position can thus be considered in addition to internship and engineering positions.

Opportunities

  • Master Cython and type inference
  • Master Python's PyObject meta-object protocol
  • Master multi-threaded garbage collection
  • Master object oriented concurrent programming (OOCP)
  • Master the Actor concurrent programming model

Responsibilities

  • Contribute to the Cython compiler
  • Contribute to research projects to build the future of our open source stack
  • Understand how multi-threaded runtimes operate
  • Accelerate Python execution x1000
  • Create a PyObject model implementation without GIL
  • Accelerate large Python code bases automatically through type inference
  • Improve the Python language competitive advantage
  • Contribute to open source projects such as Wendelin, Pandas, NumPy, scikit-learn, SlapOS, NEO, etc.
  • Contribute to research projects to build the future of our open source stack

Requirements

  • Passionate, self-driven.
  • Willingness to contribute to an open source ecosystem and the Free Software community.
  • Good skills in GNU/Linux operating system.
  • Very good programming skills in Python and C
  • Very good software development skills (version control, testing, debugging).
  • Good command of English.

References

About Nexedi

Nexedi has been developing free software since launching in 2001. We are maintaining software solutions (see our full stack) with over 10 million lines of code including:

  • ERP5 - ERP/CRM/DMS/e-business
  • Slapos - Cloud Orchestration and deployment
  • Wendelin - Big Data/Machine Learning
  • Neo - Distributed Storage
  • Resist - Resilient Mesh Network
  • Renderjs - Promise based component framework
  • jIO - Virtual File System and storage connector
  • NayuOS - Private OS
  • OfficeJs - Private, offline capable productivity AppStore

Besides participating in various research initiatives, Nexedi provides customisation services for solutions with implementations being used by corporations such as Airbus, Sanef, Mitsubishi all around the world. We follow the principles of reflexive programming, enforce strict unit testing and emphasise using the latest web technologies.

Our economic model requires each developer to fulfill R&D objectives aiming at delivering short to medium sized solutions to customer requirements and progressing the evolution of our software stack. This way we try to be innovative and fund long term free software without the need for venture capital.

https://www.nexedi.com/NXD-Presentation.Status.Roadmap?portal_skin=Slide#/ https://www.nexedi.com/NXD-Presentation.Status.Roadmap


… or you can apply by emailing jobs@nexedi.com.