@x-python/core ·

A complete solution for python-in-browser. (check the Usage section :point_down:)

---

🔥 A REPL powered by xPython as React component is coming soon
⌛️ It's still in beta testing

---

Synopsis

Clean API to execute Python code, get code completions, format the code, install packages, and many more.

Motivation

In the past few years we used python in production browser-based applications in quite different scenarios. From just executing a python code to implementing autoformat and autocomplete. And while all of this is possible there are many questionable situations in the implementations. For example, it's been a while since we've had a full Python distribution for the browser based on webassembly - pyodide. Pyodide is great! You can just install and use it. But to use it correctly, instead of installing it in the main (UI) thread, it would be desirable to install/run it in a separate thread. Once you've created a separate thread you need to create a channel between main/UI and pyodide worker and come up with some protocol for communication. You also need to do something with error handling, handling standard output streams, non-sequential executions and other all possible corner cases. This was one of the things that xPython will handle for you :slightly_smiling_face: It also provides a clean API for code completion, installing packages, and formatting the code... and many more are coming soon. Long story short I tried to provide a clean and complete interface to interact with Python in browser-based applications.

Documentation

Contents

• Installation
• Usage
• API
  • .init
  • .exec
  • .complete
    • .repl
  • .format
  • .install
• Development

Installation

1npm install @x-python/core

or

1yarn add @x-python/core

Usage

1import * as xPython from '@x-python/core';
2
3// initialize xPython
4await xPython.init();
5
6// execute python code
7await xPython.exec({ code: '1 + 1' });
8await xPython.exec({ code: 'print("test")' });
9
10// multiline example
11await xPython.exec({
12  code: `
13import sys
14sys.version
15`,
16});
17
18// you can use built-in packages without additionally installing them
19await xPython.exec({
20  code: `
21import numpy as np
22np.random.rand()
23`,
24});
25
26// code completion
27await xPython.complete.repl({ code: `import sys; sys.ver` });
28
29// specify the cursor position
30await xPython.complete.repl({
31  code: `
32from math import factorial
33
34test = 8
35print(tes)
36factorial(x)
37`,
38  line: 5,
39  column: 9,
40});
41
42// format the code
43const { result } = await xPython.format({
44  code: `
45def add(a,            b):
46  return a +        b
47
48print(add(12,
49
5054))
51`,
52});
53
54console.log(result);
55
56// install packages
57await xPython.install(['nicelog']);
58
59// and use the newly installed package :)
60const { stderr } = await xPython.exec({
61  code: `
62import logging
63import sys
64
65from nicelog.formatters import Colorful
66
67# Setup a logger
68logger = logging.getLogger('foo')
69logger.setLevel(logging.DEBUG)
70
71# Setup a handler, writing colorful output
72# to the console
73handler = logging.StreamHandler(sys.stderr)
74handler.setFormatter(Colorful())
75handler.setLevel(logging.DEBUG)
76logger.addHandler(handler)
77
78# Now log some messages..
79logger.debug('Debug message')
80logger.info('Info message')
81logger.warning('Warning message')
82logger.error('Error message')
83logger.critical('Critical message')
84try:
85    raise ValueError('This is an exception')
86except:
87    logger.exception("An error occurred")
88`,
89});
90
91console.log(stderr);

API

.init

It will initialize xPython. Most importantly it will create a separate thread (dedicated web worker), install pyodide inside that thread, create a channel, and setup all necessary packages and functions for further usage.

1import * as xPython from '@x-python/core';
2
3await xPython.init();

Usually, we do initialize xPython before using other methods (like exec, complete, etc), but it's not mandatory :slightly_smiling_face: So, you can go ahead and do xPython.exec({ code: '...' }) without doing xPython.init() - it will do xPython.init() on first xPython.exec (or .complete, .format and any other supported method) call if it's not initialized. The aim of the existence of a separate initialize method is to provide full flexibility to developers. The initialization process takes time and it should be possible to handle that time in the way you want. So, you can do await xPython.init(); at the beginning or do it after a certain user action or, if you are okay with your users waiting a little bit more after the first execution then you can skip the initialization process and it will be handled automatically :slightly_smiling_face:

.exec

exec is one of the most frequently used. Basically, it's for executing python code. A simple usage looks like this:

1import * as xPython from '@x-python/core';
2
3await xPython.exec({ code: '1 + 1' });

You can also provide a context with global variables, like:

1import * as xPython from '@x-python/core';
2
3await xPython.exec({ code: 'x + 1', context: { x: 1 } });

But let's take a closer look at what it returns. In both cases we will get something like this:

1{ result: 2, error: null, stdout: '', stderr: '' }

result is what is returned from the executed script. But we also have stdout (and stderr) for standard output streams. If we execute print("test") nothing will be returned, but you will have a stdout.

1import * as xPython from '@x-python/core';
2
3await xPython.exec({ code: 'print("test")' });
4
5// { result: undefined, error: null, stdout: 'test', stderr: '' }

Of course you can exec multiline code snippets:

1import * as xPython from '@x-python/core';
2
3await xPython.exec({
4  code: `
5import sys
6
7sys.version
8`,
9});

You can directly use pyodide built-in package list without installing them. The full list is here

1import * as xPython from '@x-python/core';
2
3await xPython.exec({
4  code: `
5import numpy as np
6
7np.random.rand()
8`,
9});

It will autodetect numpy and it will install it if you're using it for the first time.

.complete

To get code completions you can use xPython.complete.repl. Simple usage looks like this:

1import * as xPython from '@x-python/core';
2
3await xPython.complete.repl({ code: 'import sys; sys.ver' });

This example will return an array with two possible options: version and version_info :slightly_smiling_face:

The full signature of this method also includes line and column options to specify the cursor position. If line isn't provided xPython will assume it's the last line and, correspondingly, if column isn't provided it will assume that it's the last column. So, in the previous example, it assumed that cursor is at the end of sys.var and returned code completions based on that assumption. An example with cursor position specified:

1await xPython.complete.repl({
2  code: `
3from math import factorial
4
5test = 8
6print(tes)
7factorial(x)
8`,
9  line: 5,
10  column: 9,
11});

This will return the only available option here: test.

The curious eye may notice that instead of .complete we called .complete.repl. When it comes to code completion at least two environments can be your target: REPL and Script/File/Editor. And based on the environment code completion can vary. In the current version, we do support only REPL, but very soon other options will also be available ⏳

.format

Code formatting is an essential part of interacting with your code. A simple usage looks like this:

1import * as xPython from '@x-python/core';
2
3const { result } = await xPython.format({
4  code: `
5def add(a,            b):
6  return a +        b
7
8print(add(12,
9
1054))
11`,
12});
13
14console.log(result);

NOTE: in upcoming versions a full configuration option will be provided.

.install

The entire standard library is available out of the box, so you can import sys, math, or os without doing anything special. In addition to this pyodide also provides a list of built-in packages, like numpy, pandas, scipy, matplotlib, scikit-learn, etc. Check the full list here. You can use any package from the above-mentioned list and it will be installed automatically and on demand. And if that's not enough you can still install any pure Python packages with wheels available on PyPI :slightly_smiling_face: Let's install the package called nicelog.

1import * as xPython from '@x-python/core';
2
3await xPython.install(['nicelog']);

That's it :slightly_smiling_face: Now you have nicelog installed and it's ready to be used. Not familiar with nicelog? Let's check what's inside:

1import * as xPython from '@x-python/core';
2
3await xPython.complete.repl({ code: 'from nicelog import ' });

As it's already installed it should be available for code completion as well :white_check_mark:

Example from the nicelog PyPI page.

1const { stderr } = await xPython.exec({
2  code: `
3import logging
4import sys
5
6from nicelog.formatters import Colorful
7
8# Setup a logger
9logger = logging.getLogger('foo')
10logger.setLevel(logging.DEBUG)
11
12# Setup a handler, writing colorful output
13# to the console
14handler = logging.StreamHandler(sys.stderr)
15handler.setFormatter(Colorful())
16handler.setLevel(logging.DEBUG)
17logger.addHandler(handler)
18
19# Now log some messages..
20logger.debug('Debug message')
21logger.info('Info message')
22logger.warning('Warning message')
23logger.error('Error message')
24logger.critical('Critical message')
25try:
26    raise ValueError('This is an exception')
27except:
28    logger.exception("An error occurred")
29`,
30});
31
32console.log(stderr);

Development

To play with the library locally do the following steps:

1. clone this repo

1git clone git@github.com:suren-atoyan/x-python.git

2. install dependencies

1npm install # or yarn

3. run the dev server

1npm run dev

That's it :slightly_smiling_face: Under /playground folder you can find the index.html file which contains a script with the demo code and under /src folder you can find the library source code. Enjoy it :tada:

License

MIT