Compare commits

..

10 Commits

Author SHA1 Message Date
3wc
e1f2318273 Merge branch 'tests' into publicapi-tests 2021-07-17 13:37:43 +02:00
3wc
7b9e9debab Disable VM creation check for the moment 2021-07-17 13:37:28 +02:00
3wc
bef379273f Fix botched merge 2021-07-17 13:32:54 +02:00
3wc
82c0c63ff4 Add tests for public API 2021-07-17 13:29:16 +02:00
3wc
d5e0e069d1 Merge branch 'tests' into publicapi-tests 2021-07-17 13:28:50 +02:00
3wc
b8d149e862 Updates for upstream IP handling 2021-07-17 13:23:43 +02:00
3wc
cf42ac5e4d Add basic "create" API..
.. using server-side API tokens
2021-07-17 12:18:16 +02:00
3wc
9823bda4a2 Add SSH key tests 2021-07-17 12:10:28 +02:00
3wc
4c7caa1a38 Initial console tests
NB capsul create isn't working properly, see #83
2021-07-17 11:54:15 +02:00
3wc
665e087bd4 Basic testing using flask-testing
This commit makes it possible to override settings during tests, by
switching capsulflask/__init__.py to a "create_app" pattern, and using
`dotenv_values` instead of `load_dotenv`.

The create_app() method returns a Flask app instance, to give
more control over when to initialise the app. This allows setting
environment variables in test files.

Then, use dotenv_values to override loaded .env variables with ones from
the environment, so that tests can set `POSTGRES_CONNECTION_PARAMETERS`
and `SPOKE_MODEL` (possibly others in future..).

Inital tests for the "landing" pages, and login / activation, are
included.
2021-07-17 10:38:04 +02:00
57 changed files with 955 additions and 1253 deletions

View File

@ -1,13 +0,0 @@
---
kind: pipeline
name: publish docker image
steps:
- name: build and publish
image: plugins/docker
settings:
username:
from_secret: docker_reg_username_3wc
password:
from_secret: docker_reg_passwd_3wc
repo: 3wordchant/capsul-flask
tags: ${DRONE_COMMIT_BRANCH}

1
.gitignore vendored
View File

@ -1,6 +1,5 @@
notes.txt notes.txt
.env .env
.env.bak
.vscode .vscode
*.pyc *.pyc

View File

@ -1,48 +0,0 @@
FROM python:3.8-alpine as build
RUN apk add --no-cache \
build-base \
gcc \
gettext \
git \
jpeg-dev \
libffi-dev \
libjpeg \
musl-dev \
postgresql-dev \
python3-dev \
zlib-dev
RUN mkdir -p /app/{code,venv}
WORKDIR /app/code
COPY Pipfile Pipfile.lock /app/code/
RUN python3 -m venv /app/venv
RUN pip install pipenv setuptools
ENV PATH="/app/venv/bin:$PATH" VIRTUAL_ENV="/app/venv"
RUN pip install wheel cppy
# Install dependencies into the virtual environment with Pipenv
RUN pipenv install --deploy --verbose
FROM python:3.8-alpine
RUN apk add --no-cache \
cloud-utils \
libjpeg \
libpq \
libstdc++ \
libvirt-client \
openssh-client \
virt-install
COPY . /app/code/
WORKDIR /app/code
COPY --from=build /app/venv /app/venv
ENV PATH="/app/venv/bin:$PATH" VIRTUAL_ENV="/app/venv"
CMD ["gunicorn", "--bind", "0.0.0.0:5000", "-k", "gevent", "--worker-connections", "1000", "app:app"]
VOLUME /app/code
EXPOSE 5000

View File

@ -9,6 +9,7 @@ blinker = "==1.4"
click = "==7.1.2" click = "==7.1.2"
Flask = "==1.1.2" Flask = "==1.1.2"
Flask-Mail = "==0.9.1" Flask-Mail = "==0.9.1"
Flask-Testing = "==0.8.1"
gunicorn = "==20.0.4" gunicorn = "==20.0.4"
isort = "==4.3.21" isort = "==4.3.21"
itsdangerous = "==1.1.0" itsdangerous = "==1.1.0"

252
README.md
View File

@ -1,51 +1,227 @@
# capsul-flask # capsulflask
![screenshot of capsul.org home page](./docs/capsul.webp) Python Flask web application for capsul.org
Python Flask web application implementing user accounts, payment, and virtual machine management for a smol "virtual machine (vm) as a service" aka "cloud compute" provider. Originally developed by [Cyberia Computer Club](https://cyberia.club) for https://capsul.org
`capsul-flask` integrates with [Stripe](https://stripe.com/) as a credit card processor, and [BTCPay Server](https://github.com/btcpayserver/btcpayserver-docker) as a cryptocurrency payment processor. ## how to run locally
`capsul-flask` invokes [shell-scripts](./capsulflask/shell_scripts/) to create/manage [libvirt/qemu](https://www.libvirt.org/manpages/virsh.html) vms, and it depends on `dnsmasq` to act as the DHCP server for the vms. Ensure you have the pre-requisites for the psycopg2 Postgres database adapter package
`capsul-flask` has a ["hub and spoke" architecture](./architecture.md). The "Hub" runs the web application and talks to the Postrges database, while the "Spoke"(s) are responsible for creating/managing virtual machines. In this way, capsul can be scaled to span more than one machine. One instance of the capsul-flask application can run in both hub mode and spoke mode at the same time, however there must only be one instance of the app running in "Hub" mode at any given time.
## Quickstart (run capsul-flask on your computer in development mode)
``` ```
# get an instance of postgres running locally on port 5432 sudo apt install python3-dev libpq-dev
# (you don't have to use docker, but we thought this might be the easiest for a how-to example) pg_config --version
docker run --rm -it -e POSTGRES_PASSWORD=dev -p 5432:5432 postgres & ```
# install dependencies Ensure you have the wonderful `pipenv` python package management and virtual environment cli
sudo apt install pipenv python3-dev libpq-dev
# download and run ```
git clone https://giit.cyberia.club/~forest/capsul-flask sudo apt install pipenv
cd capsul-flask ```
Create python virtual environment and install packages
```
# install deps
pipenv install pipenv install
```
Run an instance of Postgres (I used docker for this, you can use whatever you want, point is its listening on localhost:5432)
```
docker run --rm -it -e POSTGRES_PASSWORD=dev -p 5432:5432 postgres
```
Run the app
```
pipenv run flask run pipenv run flask run
``` ```
Interested in learning more? How about a trip to the the `docs/` folder: Run the app in gunicorn:
- [**Setting up capsul-flask locally**](./docs/local-set-up.md) ```
- [Manually](./docs/local-set-up.md#manually) pipenv run gunicorn --bind 127.0.0.1:5000 -k gevent --worker-connections 1000 app:app
- [With docker-compose](./docs/local-set-up.md#docker_compose) ```
- [**Configuring `capsul-flask`**](./docs/configuration.md)
- [Example configuration from capsul.org (production)](./docs/configuration.md#example) Once you log in for the first time, you will want to give yourself some free capsulbux so you can create fake capsuls for testing.
- [Configuration-type-stuff that lives in the database ](./docs/configuration.md#config_that_lives_in_db)
- [Loading variables from files (docker secrets)](./docs/configuration.md#docker_secrets) Note that by default when running locally, the `SPOKE_MODEL` is set to `mock`, meaning that it won't actually try to spawn vms.
- [**`capsul-flask`'s relationship to its Database Server**](./docs/database.md)
- [Database schema management (schema versions)](./docs/database.md#schema_management) ```
- [Running manual database queries](./docs/database.md#manual_queries) pipenv run flask cli sql -c "INSERT INTO payments (email, dollars) VALUES ('<your email address here>', 20.00)"
- [**`capsul-flask`'s hub-and-spoke architecture**](./docs/architecture.md) ```
- [**Deploying capsul-flask on a server**](./docs/deployment.md)
- [Installing prerequisites for Spoke Mode](./docs/deployment.md#spoke_mode_prerequisites) ## configuration:
- [Deploying capsul-flask manually](./docs/deployment.md#deploy_manually)
- [Deploying capsul-flask with coop-cloud's docker-swarm configuration](./docs/deployment.md#coop_cloud_docker) Create a `.env` file to set up the application configuration:
- [Deploying capsul-flask with coop-cloud's `abra` deployment tool](./docs/deployment.md#coop_cloud_abra)
- [**Accepting cryptocurrency payments with BTCPay Server**](./docs/btcpay.md) ```
- [Setting up the BTCPAY_PRIVATE_KEY](./docs/btcpay.md#BTCPAY_PRIVATE_KEY) nano .env
- [Testing cryptocurrency payments](./docs/btcpay.md#testing) ```
- [Sequence diagram explaining how BTC payment process works (how we accept 0-confirmation transactions 😀)](./docs/btcpay.md#0_conf_diagram)
You can enter any environment variables referenced in `__init__.py` to this file.
For example you may enter your SMTP credentials like this:
```
MAIL_USERNAME=forest@nullhex.com
MAIL_DEFAULT_SENDER=forest@nullhex.com
MAIL_PASSWORD=**************
```
## how to view the logs on the database server (legion.cyberia.club)
`sudo -u postgres pg_dump capsul-flask | gzip -9 > capsul-backup-2021-02-15.gz`
-----
## cli
You can manually mess around with the database like this:
```
pipenv run flask cli sql -f test.sql
```
```
pipenv run flask cli sql -c 'SELECT * FROM vms'
```
This one selects the vms table with the column name header:
```
pipenv run flask cli sql -c "SELECT string_agg(column_name::text, ', ') from information_schema.columns WHERE table_name='vms'; SELECT * from vms"
```
How to modify a payment manually, like if you get a chargeback or to fix customer payment issues:
```
$ pipenv run flask cli sql -c "SELECT id, created, email, dollars, invalidated from payments"
1, 2020-05-05T00:00:00, forest.n.johnson@gmail.com, 20.00, FALSE
$ pipenv run flask cli sql -c "UPDATE payments SET invalidated = True WHERE id = 1"
1 rows affected.
$ pipenv run flask cli sql -c "SELECT id, created, email, dollars, invalidated from payments"
1, 2020-05-05T00:00:00, forest.n.johnson@gmail.com, 20.00, TRUE
```
How you would kick off the scheduled task:
```
pipenv run flask cli cron-task
```
-----
## postgres database schema management
capsulflask has a concept of a schema version. When the application starts, it will query the database for a table named
`schemaversion` that has one row and one column (`version`). If the `version` it finds is not equal to the `desiredSchemaVersion` variable set in `db.py`, it will run migration scripts from the `schema_migrations` folder one by one until the `schemaversion` table shows the correct version.
For example, the script named `02_up_xyz.sql` should contain code that migrates the database from schema version 1 to schema version 2. Likewise, the script `02_down_xyz.sql` should contain code that migrates from schema version 2 back to schema version 1.
**IMPORTANT: if you need to make changes to the schema, make a NEW schema version. DO NOT EDIT the existing schema versions.**
In general, for safety, schema version upgrades should not delete data. Schema version downgrades will simply throw an error and exit for now.
-----
## hub-and-spoke architecture
![](readme/hub-and-spoke1.png)
This diagram was created with https://app.diagrams.net/.
To edit it, download the <a download href="readme/hub-and-spoke.xml">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.
right now I have 2 types of operations, immediate mode and async.
both types of operations do assignment synchronously. so if the system cant assign the operation to one or more hosts (spokes),
or whatever the operation requires, then it will fail.
some operations tolerate partial failures, like, `capacity_avaliable` will succeed if at least one spoke succeeds.
for immediate mode requests (like `list`, `capacity_avaliable`, `destroy`), assignment and completion of the operation are the same thing.
for async ones, they can be assigned without knowing whether or not they succeeded (`create`).
![](readme/hub-and-spoke2.png)
This diagram was created with https://app.diagrams.net/.
To edit it, download the <a download href="readme/hub-and-spoke.xml">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.
if you issue a create, and it technically could go to any number of hosts, but only one host responds, it will succeed
but if you issue a create and somehow 2 hosts both think they own that task, it will fail and throw a big error. cuz it expects exactly 1 to own the create task
currently its not set up to do any polling. its not really like a queue at all. It's all immediate for the most part
-----
## how to setup btcpay server
Generate a private key and the accompanying bitpay SIN for the btcpay API client.
I used this code as an example: https://github.com/bitpay/bitpay-python/blob/master/bitpay/key_utils.py#L6
```
$ pipenv run python ./readme/generate_btcpay_keys.py
```
It should output something looking like this:
```
-----BEGIN EC PRIVATE KEY-----
EXAMPLEIArx/EXAMPLEKH23EXAMPLEsYXEXAMPLE5qdEXAMPLEcFHoAcEXAMPLEK
oUQDQgAEnWs47PT8+ihhzyvXX6/yYMAWWODluRTR2Ix6ZY7Z+MV7v0W1maJzqeqq
NQ+cpBvPDbyrDk9+Uf/sEaRCma094g==
-----END EC PRIVATE KEY-----
EXAMPLEwzAEXAMPLEEXAMPLEURD7EXAMPLE
```
In order to register the key with the btcpay server, you have to first generate a pairing token using the btcpay server interface.
This requires your btcpay server account to have access to the capsul store. Ask Cass about this.
Navigate to `Manage store: Access Tokens` at: `https://btcpay.cyberia.club/stores/<store-id>/Tokens`
![](readme/btcpay_sin_pairing.jpg)
Finally, send an http request to the btcpay server to complete the pairing:
```
curl -H "Content-Type: application/json" https://btcpay.cyberia.club/tokens -d "{'id': 'EXAMPLEwzAEXAMPLEEXAMPLEURD7EXAMPLE', 'pairingCode': 'XXXXXXX'}"
```
It should respond with a token:
```
{"data":[{"policies":[],"pairingCode":"XXXXXXX","pairingExpiration":1589473817597,"dateCreated":1589472917597,"facade":"merchant","token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","label":"capsulflask"}]}
```
And you should see the token in the btcpay server UI:
![](readme/paired.jpg)
Now simply set your `BTCPAY_PRIVATE_KEY` variable in `.env`
NOTE: make sure to use single quotes and replace the new lines with \n.
```
BTCPAY_PRIVATE_KEY='-----BEGIN EC PRIVATE KEY-----\nEXAMPLEIArx/EXAMPLEKH23EXAMPLEsYXEXAMPLE5qdEXAMPLEcFHoAcEXAMPLEK\noUQDQgAEnWs47PT8+ihhzyvXX6/yYMAWWODluRTR2Ix6ZY7Z+MV7v0W1maJzqeqq\nNQ+cpBvPDbyrDk9+Uf/sEaRCma094g==\n-----END EC PRIVATE KEY-----'
```
-----
## testing cryptocurrency payments
I used litecoin to test cryptocurrency payments, because its the simplest & lowest fee cryptocurrency that BTCPay server supports. You can download the easy-to-use litecoin SPV wallet `electrum-ltc` from [github.com/pooler/electrum-ltc](https://github.com/pooler/electrum-ltc) or [electrum-ltc.org](https://electrum-ltc.org/), set up a wallet, and then either purchase some litecoin from an exchange, or ask Forest for some litecoin to use for testing.
## sequence diagram explaining how BTC payment process works
![btcpayment_process](readme/btcpayment_process.png)
This diagram was created with https://app.diagrams.net/.
To edit it, download the <a download href="readme/btcpayment_process.drawio">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.

4
app.py
View File

@ -1,2 +1,4 @@
from capsulflask import app from capsulflask import create_app
create_app()

View File

@ -2,14 +2,13 @@ import logging
from logging.config import dictConfig as logging_dict_config from logging.config import dictConfig as logging_dict_config
import atexit import atexit
import jinja2
import os import os
import hashlib import hashlib
import requests import requests
import sys import sys
import stripe import stripe
from dotenv import load_dotenv, find_dotenv from dotenv import find_dotenv, dotenv_values
from flask import Flask from flask import Flask
from flask_mail import Mail, Message from flask_mail import Mail, Message
from flask import render_template from flask import render_template
@ -23,79 +22,68 @@ from capsulflask import hub_model, spoke_model, cli
from capsulflask.btcpay import client as btcpay from capsulflask.btcpay import client as btcpay
from capsulflask.http_client import MyHTTPClient from capsulflask.http_client import MyHTTPClient
class StdoutMockFlaskMail: class StdoutMockFlaskMail:
def send(self, message: Message): def send(self, message: Message):
current_app.logger.info(f"Email would have been sent if configured:\n\nto: {','.join(message.recipients)}\nsubject: {message.subject}\nbody:\n\n{message.body}\n\n") current_app.logger.info(f"Email would have been sent if configured:\n\nto: {','.join(message.recipients)}\nsubject: {message.subject}\nbody:\n\n{message.body}\n\n")
def create_app():
load_dotenv(find_dotenv()) config = {
**dotenv_values(find_dotenv()),
**os.environ, # override loaded values with environment variables
}
for var_name in [ app = Flask(__name__)
"SPOKE_HOST_TOKEN", "HUB_TOKEN", "STRIPE_SECRET_KEY",
"BTCPAY_PRIVATE_KEY", "MAIL_PASSWORD"
]:
var = os.environ.get(f"{var_name}_FILE")
if not var:
continue
if not os.path.isfile(var): app.config.from_mapping(
continue TESTING=config.get("TESTING", False),
BASE_URL=config.get("BASE_URL", "http://localhost:5000"),
with open(var) as secret_file: SECRET_KEY=config.get("SECRET_KEY", "dev"),
os.environ[var_name] = secret_file.read().rstrip('\n') HUB_MODE_ENABLED=config.get("HUB_MODE_ENABLED", "True").lower() in ['true', '1', 't', 'y', 'yes'],
del os.environ[f"{var_name}_FILE"] SPOKE_MODE_ENABLED=config.get("SPOKE_MODE_ENABLED", "True").lower() in ['true', '1', 't', 'y', 'yes'],
INTERNAL_HTTP_TIMEOUT_SECONDS=config.get("INTERNAL_HTTP_TIMEOUT_SECONDS", "300"),
app = Flask(__name__) HUB_MODEL=config.get("HUB_MODEL", "capsul-flask"),
SPOKE_MODEL=config.get("SPOKE_MODEL", "mock"),
app.config.from_mapping( LOG_LEVEL=config.get("LOG_LEVEL", "INFO"),
BASE_URL=os.environ.get("BASE_URL", default="http://localhost:5000"), SPOKE_HOST_ID=config.get("SPOKE_HOST_ID", "baikal"),
SECRET_KEY=os.environ.get("SECRET_KEY", default="dev"), SPOKE_HOST_TOKEN=config.get("SPOKE_HOST_TOKEN", "changeme"),
HUB_MODE_ENABLED=os.environ.get("HUB_MODE_ENABLED", default="True").lower() in ['true', '1', 't', 'y', 'yes'], HUB_TOKEN=config.get("HUB_TOKEN", "changeme"),
SPOKE_MODE_ENABLED=os.environ.get("SPOKE_MODE_ENABLED", default="True").lower() in ['true', '1', 't', 'y', 'yes'],
INTERNAL_HTTP_TIMEOUT_SECONDS=os.environ.get("INTERNAL_HTTP_TIMEOUT_SECONDS", default="300"),
HUB_MODEL=os.environ.get("HUB_MODEL", default="capsul-flask"),
SPOKE_MODEL=os.environ.get("SPOKE_MODEL", default="mock"),
LOG_LEVEL=os.environ.get("LOG_LEVEL", default="INFO"),
SPOKE_HOST_ID=os.environ.get("SPOKE_HOST_ID", default="baikal"),
SPOKE_HOST_TOKEN=os.environ.get("SPOKE_HOST_TOKEN", default="changeme"),
HUB_TOKEN=os.environ.get("HUB_TOKEN", default="changeme"),
# https://www.postgresql.org/docs/9.1/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS # https://www.postgresql.org/docs/9.1/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS
# https://stackoverflow.com/questions/56332906/where-to-put-ssl-certificates-when-trying-to-connect-to-a-remote-database-using # https://stackoverflow.com/questions/56332906/where-to-put-ssl-certificates-when-trying-to-connect-to-a-remote-database-using
# TLS example: sslmode=verify-full sslrootcert=letsencrypt-root-ca.crt host=db.example.com port=5432 user=postgres password=dev dbname=postgres # TLS example: sslmode=verify-full sslrootcert=letsencrypt-root-ca.crt host=db.example.com port=5432 user=postgres password=dev dbname=postgres
POSTGRES_CONNECTION_PARAMETERS=os.environ.get( POSTGRES_CONNECTION_PARAMETERS=config.get(
"POSTGRES_CONNECTION_PARAMETERS", "POSTGRES_CONNECTION_PARAMETERS",
default="host=localhost port=5432 user=postgres password=dev dbname=postgres" "host=localhost port=5432 user=postgres password=dev dbname=postgres"
), ),
DATABASE_SCHEMA=os.environ.get("DATABASE_SCHEMA", default="public"), DATABASE_SCHEMA=config.get("DATABASE_SCHEMA", "public"),
MAIL_SERVER=os.environ.get("MAIL_SERVER", default=""), MAIL_SERVER=config.get("MAIL_SERVER", ""),
MAIL_PORT=os.environ.get("MAIL_PORT", default="465"), MAIL_PORT=config.get("MAIL_PORT", "465"),
MAIL_USE_TLS=os.environ.get("MAIL_USE_TLS", default="False").lower() in ['true', '1', 't', 'y', 'yes'], MAIL_USE_TLS=config.get("MAIL_USE_TLS", "False").lower() in ['true', '1', 't', 'y', 'yes'],
MAIL_USE_SSL=os.environ.get("MAIL_USE_SSL", default="True").lower() in ['true', '1', 't', 'y', 'yes'], MAIL_USE_SSL=config.get("MAIL_USE_SSL", "True").lower() in ['true', '1', 't', 'y', 'yes'],
MAIL_USERNAME=os.environ.get("MAIL_USERNAME", default=""), MAIL_USERNAME=config.get("MAIL_USERNAME", ""),
MAIL_PASSWORD=os.environ.get("MAIL_PASSWORD", default=""), MAIL_PASSWORD=config.get("MAIL_PASSWORD", ""),
MAIL_DEFAULT_SENDER=os.environ.get("MAIL_DEFAULT_SENDER", default="no-reply@capsul.org"), MAIL_DEFAULT_SENDER=config.get("MAIL_DEFAULT_SENDER", "no-reply@capsul.org"),
ADMIN_EMAIL_ADDRESSES=os.environ.get("ADMIN_EMAIL_ADDRESSES", default="ops@cyberia.club"), ADMIN_EMAIL_ADDRESSES=config.get("ADMIN_EMAIL_ADDRESSES", "ops@cyberia.club"),
ADMIN_PANEL_ALLOW_EMAIL_ADDRESSES=os.environ.get("ADMIN_PANEL_ALLOW_EMAIL_ADDRESSES", default="forest.n.johnson@gmail.com,capsul@cyberia.club"), ADMIN_PANEL_ALLOW_EMAIL_ADDRESSES=config.get("ADMIN_PANEL_ALLOW_EMAIL_ADDRESSES", "forest.n.johnson@gmail.com,capsul@cyberia.club"),
PROMETHEUS_URL=os.environ.get("PROMETHEUS_URL", default="https://prometheus.cyberia.club"), PROMETHEUS_URL=config.get("PROMETHEUS_URL", "https://prometheus.cyberia.club"),
STRIPE_API_VERSION=os.environ.get("STRIPE_API_VERSION", default="2020-03-02"), STRIPE_API_VERSION=config.get("STRIPE_API_VERSION", "2020-03-02"),
STRIPE_SECRET_KEY=os.environ.get("STRIPE_SECRET_KEY", default=""), STRIPE_SECRET_KEY=config.get("STRIPE_SECRET_KEY", ""),
STRIPE_PUBLISHABLE_KEY=os.environ.get("STRIPE_PUBLISHABLE_KEY", default=""), STRIPE_PUBLISHABLE_KEY=config.get("STRIPE_PUBLISHABLE_KEY", ""),
#STRIPE_WEBHOOK_SECRET=os.environ.get("STRIPE_WEBHOOK_SECRET", default="") #STRIPE_WEBHOOK_SECRET=config.get("STRIPE_WEBHOOK_SECRET", "")
BTCPAY_PRIVATE_KEY=os.environ.get("BTCPAY_PRIVATE_KEY", default="").replace("\\n", "\n"), BTCPAY_PRIVATE_KEY=config.get("BTCPAY_PRIVATE_KEY", "").replace("\\n", "\n"),
BTCPAY_URL=os.environ.get("BTCPAY_URL", default=""), BTCPAY_URL=config.get("BTCPAY_URL", "https://btcpay.cyberia.club")
)
THEME=os.environ.get("THEME", default="") app.config['HUB_URL'] = config.get("HUB_URL", app.config['BASE_URL'])
)
app.config['HUB_URL'] = os.environ.get("HUB_URL", default=app.config['BASE_URL']) class SetLogLevelToDebugForHeartbeatRelatedMessagesFilter(logging.Filter):
class SetLogLevelToDebugForHeartbeatRelatedMessagesFilter(logging.Filter):
def isHeartbeatRelatedString(self, thing): def isHeartbeatRelatedString(self, thing):
# thing_string = "<error>" # thing_string = "<error>"
is_in_string = False is_in_string = False
@ -119,7 +107,7 @@ class SetLogLevelToDebugForHeartbeatRelatedMessagesFilter(logging.Filter):
return True return True
logging_dict_config({ logging_dict_config({
'version': 1, 'version': 1,
'formatters': {'default': { 'formatters': {'default': {
'format': '[%(asctime)s] %(levelname)s in %(module)s: %(message)s', 'format': '[%(asctime)s] %(levelname)s in %(module)s: %(message)s',
@ -139,7 +127,7 @@ logging_dict_config({
'level': app.config['LOG_LEVEL'], 'level': app.config['LOG_LEVEL'],
'handlers': ['wsgi'] 'handlers': ['wsgi']
} }
}) })
# app.logger.critical("critical") # app.logger.critical("critical")
# app.logger.error("error") # app.logger.error("error")
@ -147,48 +135,40 @@ logging_dict_config({
# app.logger.info("info") # app.logger.info("info")
# app.logger.debug("debug") # app.logger.debug("debug")
stripe.api_key = app.config['STRIPE_SECRET_KEY'] stripe.api_key = app.config['STRIPE_SECRET_KEY']
stripe.api_version = app.config['STRIPE_API_VERSION'] stripe.api_version = app.config['STRIPE_API_VERSION']
if app.config['MAIL_SERVER'] != "": if app.config['MAIL_SERVER'] != "":
app.config['FLASK_MAIL_INSTANCE'] = Mail(app) app.config['FLASK_MAIL_INSTANCE'] = Mail(app)
else: else:
app.logger.warning("No MAIL_SERVER configured. capsul will simply print emails to stdout.") app.logger.warning("No MAIL_SERVER configured. capsul will simply print emails to stdout.")
app.config['FLASK_MAIL_INSTANCE'] = StdoutMockFlaskMail() app.config['FLASK_MAIL_INSTANCE'] = StdoutMockFlaskMail()
app.config['HTTP_CLIENT'] = MyHTTPClient(timeout_seconds=int(app.config['INTERNAL_HTTP_TIMEOUT_SECONDS'])) app.config['HTTP_CLIENT'] = MyHTTPClient(timeout_seconds=int(app.config['INTERNAL_HTTP_TIMEOUT_SECONDS']))
app.config['BTCPAY_ENABLED'] = False
if app.config['BTCPAY_URL'] != "":
try: try:
app.config['BTCPAY_CLIENT'] = btcpay.Client(api_uri=app.config['BTCPAY_URL'], pem=app.config['BTCPAY_PRIVATE_KEY']) app.config['BTCPAY_CLIENT'] = btcpay.Client(api_uri=app.config['BTCPAY_URL'], pem=app.config['BTCPAY_PRIVATE_KEY'])
app.config['BTCPAY_ENABLED'] = True
except: except:
app.logger.warning("unable to create btcpay client. Capsul will work fine except cryptocurrency payments will not work. The error was: " + my_exec_info_message(sys.exc_info())) app.logger.warning("unable to create btcpay client. Capsul will work fine except cryptocurrency payments will not work. The error was: " + my_exec_info_message(sys.exc_info()))
# only start the scheduler and attempt to migrate the database if we are running the app. # only start the scheduler and attempt to migrate the database if we are running the app.
# otherwise we are running a CLI command. # otherwise we are running a CLI command.
command_line = ' '.join(sys.argv) command_line = ' '.join(sys.argv)
is_running_server = ('flask run' in command_line) or ('gunicorn' in command_line) is_running_server = (
('flask run' in command_line) or
('gunicorn' in command_line) or
('test' in command_line)
)
app.logger.info(f"is_running_server: {is_running_server}") app.logger.info(f"is_running_server: {is_running_server}")
if app.config['THEME'] != "":
my_loader = jinja2.ChoiceLoader([
jinja2.FileSystemLoader(
[os.path.join('capsulflask', 'theme', app.config['THEME']),
'capsulflask/templates']),
])
app.jinja_loader = my_loader
if app.config['HUB_MODE_ENABLED']:
if app.config['HUB_MODE_ENABLED']:
if app.config['HUB_MODEL'] == "capsul-flask": if app.config['HUB_MODEL'] == "capsul-flask":
app.config['HUB_MODEL'] = hub_model.CapsulFlaskHub() app.config['HUB_MODEL'] = hub_model.CapsulFlaskHub()
# debug mode (flask reloader) runs two copies of the app. When running in debug mode, # debug mode (flask reloader) runs two copies of the app. When running in debug mode,
# we only want to start the scheduler one time. # we only want to start the scheduler one time.
if is_running_server and (not app.debug or os.environ.get('WERKZEUG_RUN_MAIN') == 'true'): if is_running_server and (not app.debug or config.get('WERKZEUG_RUN_MAIN') == 'true'):
scheduler = BackgroundScheduler() scheduler = BackgroundScheduler()
heartbeat_task_url = f"{app.config['HUB_URL']}/hub/heartbeat-task" heartbeat_task_url = f"{app.config['HUB_URL']}/hub/heartbeat-task"
heartbeat_task_headers = {'Authorization': f"Bearer {app.config['HUB_TOKEN']}"} heartbeat_task_headers = {'Authorization': f"Bearer {app.config['HUB_TOKEN']}"}
@ -204,23 +184,23 @@ if app.config['HUB_MODE_ENABLED']:
from capsulflask import db from capsulflask import db
db.init_app(app, is_running_server) db.init_app(app, is_running_server)
from capsulflask import auth, landing, console, payment, metrics, cli, hub_api, admin from capsulflask import (
auth, landing, console, payment, metrics, cli, hub_api, publicapi, admin
)
app.register_blueprint(landing.bp)
app.register_blueprint(auth.bp) app.register_blueprint(auth.bp)
app.register_blueprint(landing.bp)
app.register_blueprint(console.bp) app.register_blueprint(console.bp)
app.register_blueprint(payment.bp) app.register_blueprint(payment.bp)
app.register_blueprint(metrics.bp) app.register_blueprint(metrics.bp)
app.register_blueprint(cli.bp) app.register_blueprint(cli.bp)
app.register_blueprint(hub_api.bp) app.register_blueprint(hub_api.bp)
app.register_blueprint(admin.bp) app.register_blueprint(admin.bp)
app.register_blueprint(publicapi.bp)
app.add_url_rule("/", endpoint="index") app.add_url_rule("/", endpoint="index")
if app.config['SPOKE_MODE_ENABLED']:
if app.config['SPOKE_MODE_ENABLED']:
if app.config['SPOKE_MODEL'] == "shell-scripts": if app.config['SPOKE_MODEL'] == "shell-scripts":
app.config['SPOKE_MODEL'] = spoke_model.ShellScriptSpoke() app.config['SPOKE_MODEL'] = spoke_model.ShellScriptSpoke()
else: else:
@ -230,8 +210,8 @@ if app.config['SPOKE_MODE_ENABLED']:
app.register_blueprint(spoke_api.bp) app.register_blueprint(spoke_api.bp)
@app.after_request @app.after_request
def security_headers(response): def security_headers(response):
response.headers['X-Frame-Options'] = 'SAMEORIGIN' response.headers['X-Frame-Options'] = 'SAMEORIGIN'
if 'Content-Security-Policy' not in response.headers: if 'Content-Security-Policy' not in response.headers:
response.headers['Content-Security-Policy'] = "default-src 'self'" response.headers['Content-Security-Policy'] = "default-src 'self'"
@ -239,8 +219,8 @@ def security_headers(response):
return response return response
@app.context_processor @app.context_processor
def override_url_for(): def override_url_for():
""" """
override the url_for function built into flask override the url_for function built into flask
with our own custom implementation that busts the cache correctly when files change with our own custom implementation that busts the cache correctly when files change
@ -248,7 +228,7 @@ def override_url_for():
return dict(url_for=url_for_with_cache_bust) return dict(url_for=url_for_with_cache_bust)
def url_for_with_cache_bust(endpoint, **values): def url_for_with_cache_bust(endpoint, **values):
""" """
Add a query parameter based on the hash of the file, this acts as a cache bust Add a query parameter based on the hash of the file, this acts as a cache bust
""" """
@ -272,3 +252,5 @@ def url_for_with_cache_bust(endpoint, **values):
values['q'] = current_app.config['STATIC_FILE_HASH_CACHE'][filename] values['q'] = current_app.config['STATIC_FILE_HASH_CACHE'][filename]
return url_for(endpoint, **values) return url_for(endpoint, **values)
return app

View File

@ -1,3 +1,4 @@
from base64 import b64decode
import functools import functools
import re import re
@ -24,6 +25,15 @@ def account_required(view):
@functools.wraps(view) @functools.wraps(view)
def wrapped_view(**kwargs): def wrapped_view(**kwargs):
api_token = request.headers.get('authorization', None)
if api_token is not None:
email = get_model().authenticate_token(b64decode(api_token).decode('utf-8'))
if email is not None:
session.clear()
session["account"] = email
session["csrf-token"] = generate()
if session.get("account") is None or session.get("csrf-token") is None : if session.get("account") is None or session.get("csrf-token") is None :
return redirect(url_for("auth.login")) return redirect(url_for("auth.login"))
@ -31,7 +41,6 @@ def account_required(view):
return wrapped_view return wrapped_view
def admin_account_required(view): def admin_account_required(view):
"""View decorator that redirects non-admin users to the login page.""" """View decorator that redirects non-admin users to the login page."""

View File

@ -1,7 +1,9 @@
from base64 import b64encode
from datetime import datetime, timedelta
import json
import re import re
import sys import sys
import json
from datetime import datetime, timedelta
from flask import Blueprint from flask import Blueprint
from flask import flash from flask import flash
from flask import current_app from flask import current_app
@ -98,7 +100,6 @@ def index():
@bp.route("/<string:id>", methods=("GET", "POST")) @bp.route("/<string:id>", methods=("GET", "POST"))
@account_required @account_required
def detail(id): def detail(id):
duration=request.args.get('duration') duration=request.args.get('duration')
if not duration: if not duration:
duration = "5m" duration = "5m"
@ -188,52 +189,31 @@ def detail(id):
duration=duration duration=duration
) )
def _create(vm_sizes, operating_systems, public_keys_for_account, server_data):
@bp.route("/create", methods=("GET", "POST"))
@account_required
def create():
vm_sizes = get_model().vm_sizes_dict()
operating_systems = get_model().operating_systems_dict()
public_keys_for_account = get_model().list_ssh_public_keys_for_account(session["account"])
account_balance = get_account_balance(get_vms(), get_payments(), datetime.utcnow())
capacity_avaliable = current_app.config["HUB_MODEL"].capacity_avaliable(512*1024*1024)
errors = list() errors = list()
affordable_vm_sizes = dict() size = server_data.get("size")
for key, vm_size in vm_sizes.items(): os = server_data.get("os")
# if a user deposits $7.50 and then creates an f1-s vm which costs 7.50 a month, posted_keys_count = int(server_data.get("ssh_authorized_key_count"))
# then they have to delete the vm and re-create it, they will not be able to, they will have to pay again.
# so for UX it makes a lot of sense to give a small margin of 25 cents for usability sake
if vm_size["dollars_per_month"] <= account_balance+0.25:
affordable_vm_sizes[key] = vm_size
if request.method == "POST":
if "csrf-token" not in request.form or request.form['csrf-token'] != session['csrf-token']:
return abort(418, f"u want tea")
size = request.form["size"]
os = request.form["os"]
if not size: if not size:
errors.append("Size is required") errors.append("Size is required")
elif size not in vm_sizes: elif size not in vm_sizes:
errors.append(f"Invalid size {size}") errors.append(f"Invalid size {size}")
elif size not in affordable_vm_sizes:
errors.append(f"Your account must have enough credit to run an {size} for 1 month before you will be allowed to create it")
if not os: if not os:
errors.append("OS is required") errors.append("OS is required")
elif os not in operating_systems: elif os not in operating_systems:
errors.append(f"Invalid os {os}") errors.append(f"Invalid os {os}")
posted_keys_count = int(request.form["ssh_authorized_key_count"])
posted_keys = list() posted_keys = list()
if posted_keys_count > 1000: if posted_keys_count > 1000:
errors.append("something went wrong with ssh keys") errors.append("something went wrong with ssh keys")
else: else:
for i in range(0, posted_keys_count): for i in range(0, posted_keys_count):
if f"ssh_key_{i}" in request.form: if f"ssh_key_{i}" in server_data:
posted_name = request.form[f"ssh_key_{i}"] posted_name = server_data.get(f"ssh_key_{i}")
key = None key = None
for x in public_keys_for_account: for x in public_keys_for_account:
if x['name'] == posted_name: if x['name'] == posted_name:
@ -246,7 +226,9 @@ def create():
if len(posted_keys) == 0: if len(posted_keys) == 0:
errors.append("At least one SSH Public Key is required") errors.append("At least one SSH Public Key is required")
capacity_avaliable = current_app.config["HUB_MODEL"].capacity_avaliable(vm_sizes[size]['memory_mb']*1024*1024) capacity_avaliable = current_app.config["HUB_MODEL"].capacity_avaliable(
vm_sizes[size]['memory_mb']*1024*1024
)
if not capacity_avaliable: if not capacity_avaliable:
errors.append(""" errors.append("""
@ -255,8 +237,6 @@ def create():
if len(errors) == 0: if len(errors) == 0:
id = make_capsul_id() id = make_capsul_id()
# we can't create the vm record in the DB yet because its IP address needs to be allocated first.
# so it will be created when the allocation happens inside the hub_api.
current_app.config["HUB_MODEL"].create( current_app.config["HUB_MODEL"].create(
email = session["account"], email = session["account"],
id=id, id=id,
@ -265,11 +245,40 @@ def create():
template_image_file_name=operating_systems[os]['template_image_file_name'], template_image_file_name=operating_systems[os]['template_image_file_name'],
vcpus=vm_sizes[size]['vcpus'], vcpus=vm_sizes[size]['vcpus'],
memory_mb=vm_sizes[size]['memory_mb'], memory_mb=vm_sizes[size]['memory_mb'],
ssh_authorized_keys=list(map(lambda x: dict(name=x['name'], content=x['content']), posted_keys)) ssh_authorized_keys=list(map(lambda x: x["content"], posted_keys))
) )
return id, errors
return None, errors
@bp.route("/create", methods=("GET", "POST"))
@account_required
def create():
vm_sizes = get_model().vm_sizes_dict()
operating_systems = get_model().operating_systems_dict()
public_keys_for_account = get_model().list_ssh_public_keys_for_account(session["account"])
account_balance = get_account_balance(get_vms(), get_payments(), datetime.utcnow())
capacity_avaliable = current_app.config["HUB_MODEL"].capacity_avaliable(512*1024*1024)
errors = list()
if request.method == "POST":
if "csrf-token" not in request.form or request.form['csrf-token'] != session['csrf-token']:
return abort(418, f"u want tea")
id, errors = _create(
vm_sizes,
operating_systems,
public_keys_for_account,
request.form)
if len(errors) == 0:
return redirect(f"{url_for('console.index')}?created={id}") return redirect(f"{url_for('console.index')}?created={id}")
affordable_vm_sizes = dict()
for key, vm_size in vm_sizes.items():
# if a user deposits $7.50 and then creates an f1-s vm which costs 7.50 a month,
# then they have to delete the vm and re-create it, they will not be able to, they will have to pay again.
# so for UX it makes a lot of sense to give a small margin of 25 cents for usability sake
if vm_size["dollars_per_month"] <= account_balance+0.25:
affordable_vm_sizes[key] = vm_size
for error in errors: for error in errors:
flash(error) flash(error)
@ -290,23 +299,25 @@ def create():
vm_sizes=affordable_vm_sizes vm_sizes=affordable_vm_sizes
) )
@bp.route("/ssh", methods=("GET", "POST")) @bp.route("/keys", methods=("GET", "POST"))
@account_required @account_required
def ssh_public_keys(): def ssh_api_keys():
errors = list() errors = list()
token = None
if request.method == "POST": if request.method == "POST":
if "csrf-token" not in request.form or request.form['csrf-token'] != session['csrf-token']: if "csrf-token" not in request.form or request.form['csrf-token'] != session['csrf-token']:
return abort(418, f"u want tea") return abort(418, f"u want tea")
method = request.form["method"] action = request.form["action"]
if action == 'upload_ssh_key':
content = None content = None
if method == "POST":
content = request.form["content"].replace("\r", " ").replace("\n", " ").strip() content = request.form["content"].replace("\r", " ").replace("\n", " ").strip()
name = request.form["name"] name = request.form["name"]
if not name or len(name.strip()) < 1: if not name or len(name.strip()) < 1:
if method == "POST":
parts = re.split(" +", content) parts = re.split(" +", content)
if len(parts) > 2 and len(parts[2].strip()) > 0: if len(parts) > 2 and len(parts[2].strip()) > 0:
name = parts[2].strip() name = parts[2].strip()
@ -317,7 +328,6 @@ def ssh_public_keys():
if not re.match(r"^[0-9A-Za-z_@:. -]+$", name): if not re.match(r"^[0-9A-Za-z_@:. -]+$", name):
errors.append(f"Key name '{name}' must match \"^[0-9A-Za-z_@:. -]+$\"") errors.append(f"Key name '{name}' must match \"^[0-9A-Za-z_@:. -]+$\"")
if method == "POST":
if not content or len(content.strip()) < 1: if not content or len(content.strip()) < 1:
errors.append("Content is required") errors.append("Content is required")
else: else:
@ -330,24 +340,36 @@ def ssh_public_keys():
if len(errors) == 0: if len(errors) == 0:
get_model().create_ssh_public_key(session["account"], name, content) get_model().create_ssh_public_key(session["account"], name, content)
elif method == "DELETE": elif action == "delete_ssh_key":
if len(errors) == 0:
get_model().delete_ssh_public_key(session["account"], name) get_model().delete_ssh_public_key(session["account"], name)
elif action == "generate_api_token":
name = request.form["name"]
if name == '':
name = datetime.utcnow().strftime('%y-%m-%d %H:%M:%S')
token = b64encode(
get_model().generate_api_token(session["account"], name).encode('utf-8')
).decode('utf-8')
elif action == "delete_api_token":
get_model().delete_api_token(session["account"], request.form["id"])
for error in errors: for error in errors:
flash(error) flash(error)
keys_list=list(map( ssh_keys_list=list(map(
lambda x: dict(name=x['name'], content=f"{x['content'][:20]}...{x['content'][len(x['content'])-20:]}"), lambda x: dict(name=x['name'], content=f"{x['content'][:20]}...{x['content'][len(x['content'])-20:]}"),
get_model().list_ssh_public_keys_for_account(session["account"]) get_model().list_ssh_public_keys_for_account(session["account"])
)) ))
api_tokens_list = get_model().list_api_tokens(session["account"])
return render_template( return render_template(
"ssh-public-keys.html", "keys.html",
csrf_token = session["csrf-token"], csrf_token = session["csrf-token"],
ssh_public_keys=keys_list, api_tokens=api_tokens_list,
has_ssh_public_keys=len(keys_list) > 0 ssh_public_keys=ssh_keys_list,
generated_api_token=token,
) )
def get_vms(): def get_vms():
@ -371,7 +393,6 @@ def get_vm_months_float(vm, as_of):
return days / average_number_of_days_in_a_month return days / average_number_of_days_in_a_month
def get_account_balance(vms, payments, as_of): def get_account_balance(vms, payments, as_of):
vm_cost_dollars = 0.0 vm_cost_dollars = 0.0
for vm in vms: for vm in vms:
vm_months = get_vm_months_float(vm, as_of) vm_months = get_vm_months_float(vm, as_of)
@ -384,7 +405,6 @@ def get_account_balance(vms, payments, as_of):
@bp.route("/account-balance") @bp.route("/account-balance")
@account_required @account_required
def account_balance(): def account_balance():
payment_sessions = get_model().list_payment_sessions_for_account(session['account']) payment_sessions = get_model().list_payment_sessions_for_account(session['account'])
for payment_session in payment_sessions: for payment_session in payment_sessions:
if payment_session['type'] == 'btcpay': if payment_session['type'] == 'btcpay':
@ -426,7 +446,6 @@ def account_balance():
has_vms=len(vms_billed)>0, has_vms=len(vms_billed)>0,
vms_billed=vms_billed, vms_billed=vms_billed,
warning_text=warning_text, warning_text=warning_text,
btcpay_enabled=current_app.config["BTCPAY_ENABLED"],
payments=list(map( payments=list(map(
lambda x: dict( lambda x: dict(
dollars=x["dollars"], dollars=x["dollars"],

View File

@ -33,7 +33,7 @@ def init_app(app, is_running_server):
result = re.search(r"^\d+_(up|down)", filename) result = re.search(r"^\d+_(up|down)", filename)
if not result: if not result:
app.logger.error(f"schemaVersion {filename} must match ^\\d+_(up|down). exiting.") app.logger.error(f"schemaVersion {filename} must match ^\\d+_(up|down). exiting.")
exit(1) continue
key = result.group() key = result.group()
with open(join(schemaMigrationsPath, filename), 'rb') as file: with open(join(schemaMigrationsPath, filename), 'rb') as file:
schemaMigrations[key] = file.read().decode("utf8") schemaMigrations[key] = file.read().decode("utf8")
@ -43,7 +43,7 @@ def init_app(app, is_running_server):
hasSchemaVersionTable = False hasSchemaVersionTable = False
actionWasTaken = False actionWasTaken = False
schemaVersion = 0 schemaVersion = 0
desiredSchemaVersion = 18 desiredSchemaVersion = 19
cursor = connection.cursor() cursor = connection.cursor()
@ -128,4 +128,3 @@ def close_db(e=None):
if db_model is not None: if db_model is not None:
db_model.cursor.close() db_model.cursor.close()
current_app.config['PSYCOPG2_CONNECTION_POOL'].putconn(db_model.connection) current_app.config['PSYCOPG2_CONNECTION_POOL'].putconn(db_model.connection)

View File

@ -1,8 +1,8 @@
import re import re
# I was never able to get this type hinting to work correctly # I was never able to get this type hinting to work correctly
# from psycopg2.extensions import connection as Psycopg2Connection, cursor as Psycopg2Cursor # from psycopg2.extensions import connection as Psycopg2Connection, cursor as Psycopg2Cursor
import hashlib
from nanoid import generate from nanoid import generate
from flask import current_app from flask import current_app
from typing import List from typing import List
@ -17,7 +17,6 @@ class DBModel:
self.cursor = cursor self.cursor = cursor
# ------ LOGIN --------- # ------ LOGIN ---------
@ -44,6 +43,16 @@ class DBModel:
return (token, ignoreCaseMatches) return (token, ignoreCaseMatches)
def authenticate_token(self, token):
m = hashlib.md5()
m.update(token.encode('utf-8'))
hash_token = m.hexdigest()
self.cursor.execute("SELECT email FROM api_tokens WHERE token = %s", (hash_token, ))
result = self.cursor.fetchall()
if len(result) == 1:
return result[0]
return None
def consume_token(self, token): def consume_token(self, token):
self.cursor.execute("SELECT email FROM login_tokens WHERE token = %s and created > (NOW() - INTERVAL '20 min')", (token, )) self.cursor.execute("SELECT email FROM login_tokens WHERE token = %s and created > (NOW() - INTERVAL '20 min')", (token, ))
row = self.cursor.fetchone() row = self.cursor.fetchone()
@ -132,6 +141,32 @@ class DBModel:
self.cursor.execute( "DELETE FROM ssh_public_keys where email = %s AND name = %s", (email, name) ) self.cursor.execute( "DELETE FROM ssh_public_keys where email = %s AND name = %s", (email, name) )
self.connection.commit() self.connection.commit()
def list_api_tokens(self, email):
self.cursor.execute(
"SELECT id, token, name, created FROM api_tokens WHERE email = %s",
(email, )
)
return list(map(
lambda x: dict(id=x[0], token=x[1], name=x[2], created=x[3]),
self.cursor.fetchall()
))
def generate_api_token(self, email, name):
token = generate()
m = hashlib.md5()
m.update(token.encode('utf-8'))
hash_token = m.hexdigest()
self.cursor.execute(
"INSERT INTO api_tokens (email, name, token) VALUES (%s, %s, %s)",
(email, name, hash_token)
)
self.connection.commit()
return token
def delete_api_token(self, email, id_):
self.cursor.execute( "DELETE FROM api_tokens where email = %s AND id = %s", (email, id_))
self.connection.commit()
def list_vms_for_account(self, email): def list_vms_for_account(self, email):
self.cursor.execute(""" self.cursor.execute("""
SELECT vms.id, vms.public_ipv4, vms.public_ipv6, vms.size, vms.os, vms.created, vms.deleted, vm_sizes.dollars_per_month SELECT vms.id, vms.public_ipv4, vms.public_ipv6, vms.size, vms.os, vms.created, vms.deleted, vm_sizes.dollars_per_month
@ -479,8 +514,3 @@ class DBModel:
#cursor.close() #cursor.close()
return to_return return to_return

View File

@ -17,10 +17,6 @@ from capsulflask.http_client import HTTPResult
from capsulflask.shared import VirtualizationInterface, VirtualMachine, OnlineHost, validate_capsul_id, my_exec_info_message from capsulflask.shared import VirtualizationInterface, VirtualMachine, OnlineHost, validate_capsul_id, my_exec_info_message
class MockHub(VirtualizationInterface): class MockHub(VirtualizationInterface):
def __init__(self):
self.default_network = "public1"
self.default_ipv4 = "1.1.1.1"
def capacity_avaliable(self, additional_ram_bytes): def capacity_avaliable(self, additional_ram_bytes):
return True return True
@ -33,9 +29,9 @@ class MockHub(VirtualizationInterface):
{"key_type":"RSA", "content":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvotgzgEP65JUQ8S8OoNKy1uEEPEAcFetSp7QpONe6hj4wPgyFNgVtdoWdNcU19dX3hpdse0G8OlaMUTnNVuRlbIZXuifXQ2jTtCFUA2mmJ5bF+XjGm3TXKMNGh9PN+wEPUeWd14vZL+QPUMev5LmA8cawPiU5+vVMLid93HRBj118aCJFQxLgrdP48VPfKHFRfCR6TIjg1ii3dH4acdJAvlmJ3GFB6ICT42EmBqskz2MPe0rIFxH8YohCBbAbrbWYcptHt4e48h4UdpZdYOhEdv89GrT8BF2C5cbQ5i9qVpI57bXKrj8hPZU5of48UHLSpXG8mbH0YDiOQOfKX/Mt", "sha256":"ghee6KzRnBJhND2kEUZSaouk7CD6o6z2aAc8GPkV+GQ"}, {"key_type":"RSA", "content":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvotgzgEP65JUQ8S8OoNKy1uEEPEAcFetSp7QpONe6hj4wPgyFNgVtdoWdNcU19dX3hpdse0G8OlaMUTnNVuRlbIZXuifXQ2jTtCFUA2mmJ5bF+XjGm3TXKMNGh9PN+wEPUeWd14vZL+QPUMev5LmA8cawPiU5+vVMLid93HRBj118aCJFQxLgrdP48VPfKHFRfCR6TIjg1ii3dH4acdJAvlmJ3GFB6ICT42EmBqskz2MPe0rIFxH8YohCBbAbrbWYcptHt4e48h4UdpZdYOhEdv89GrT8BF2C5cbQ5i9qVpI57bXKrj8hPZU5of48UHLSpXG8mbH0YDiOQOfKX/Mt", "sha256":"ghee6KzRnBJhND2kEUZSaouk7CD6o6z2aAc8GPkV+GQ"},
{"key_type":"ECDSA", "content":"ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLLgOoATz9R4aS2kk7vWoxX+lshK63t9+5BIHdzZeFE1o+shlcf0Wji8cN/L1+m3bi0uSETZDOAWMP3rHLJj9Hk=", "sha256":"aCYG1aD8cv/TjzJL0bi9jdabMGksdkfa7R8dCGm1yYs"} {"key_type":"ECDSA", "content":"ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLLgOoATz9R4aS2kk7vWoxX+lshK63t9+5BIHdzZeFE1o+shlcf0Wji8cN/L1+m3bi0uSETZDOAWMP3rHLJj9Hk=", "sha256":"aCYG1aD8cv/TjzJL0bi9jdabMGksdkfa7R8dCGm1yYs"}
]""") ]""")
return VirtualMachine(id, current_app.config["SPOKE_HOST_ID"], ipv4=self.default_ipv4, ssh_host_keys=ssh_host_keys) return VirtualMachine(id, current_app.config["SPOKE_HOST_ID"], ipv4="1.1.1.1", ssh_host_keys=ssh_host_keys)
return VirtualMachine(id, current_app.config["SPOKE_HOST_ID"], ipv4=self.default_ipv4) return VirtualMachine(id, current_app.config["SPOKE_HOST_ID"], ipv4="1.1.1.1")
def list_ids(self) -> list: def list_ids(self) -> list:
return get_model().all_non_deleted_vm_ids() return get_model().all_non_deleted_vm_ids()
@ -44,16 +40,6 @@ class MockHub(VirtualizationInterface):
validate_capsul_id(id) validate_capsul_id(id)
current_app.logger.info(f"mock create: {id} for {email}") current_app.logger.info(f"mock create: {id} for {email}")
sleep(1) sleep(1)
get_model().create_vm(
email=email,
id=id,
size=size,
os=os,
host=current_app.config["SPOKE_HOST_ID"],
network_name=self.default_network,
public_ipv4=self.default_ipv4,
ssh_authorized_keys=list(map(lambda x: x["name"], ssh_authorized_keys)),
)
def destroy(self, email: str, id: str): def destroy(self, email: str, id: str):
current_app.logger.info(f"mock destroy: {id} for {email}") current_app.logger.info(f"mock destroy: {id} for {email}")
@ -63,6 +49,7 @@ class MockHub(VirtualizationInterface):
class CapsulFlaskHub(VirtualizationInterface): class CapsulFlaskHub(VirtualizationInterface):
def synchronous_operation(self, hosts: List[OnlineHost], email: str, payload: str) -> List[HTTPResult]: def synchronous_operation(self, hosts: List[OnlineHost], email: str, payload: str) -> List[HTTPResult]:
return self.generic_operation(hosts, email, payload, True)[1] return self.generic_operation(hosts, email, payload, True)[1]
@ -275,3 +262,4 @@ class CapsulFlaskHub(VirtualizationInterface):
if not result_status == "success": if not result_status == "success":
raise ValueError(f"""failed to {command} vm "{id}" on host "{host.id}" for {email}: {result_json_string}""") raise ValueError(f"""failed to {command} vm "{id}" on host "{host.id}" for {email}: {result_json_string}""")

View File

@ -12,11 +12,9 @@ def index():
@bp.route("/pricing") @bp.route("/pricing")
def pricing(): def pricing():
vm_sizes = get_model().vm_sizes_dict()
operating_systems = get_model().operating_systems_dict() operating_systems = get_model().operating_systems_dict()
return render_template( return render_template(
"pricing.html", "pricing.html",
vm_sizes=vm_sizes,
operating_systems=operating_systems operating_systems=operating_systems
) )

View File

@ -48,10 +48,6 @@ def validate_dollars():
def btcpay_payment(): def btcpay_payment():
errors = list() errors = list()
if not current_app.config['BTCPAY_ENABLED']:
flash("BTCPay is not enabled on this server")
return redirect(url_for("console.account_balance"))
if request.method == "POST": if request.method == "POST":
result = validate_dollars() result = validate_dollars()
errors = result[0] errors = result[0]

40
capsulflask/publicapi.py Normal file
View File

@ -0,0 +1,40 @@
import datetime
from flask import Blueprint
from flask import current_app
from flask import jsonify
from flask import request
from flask import session
from nanoid import generate
from capsulflask.auth import account_required
from capsulflask.db import get_model
bp = Blueprint("publicapi", __name__, url_prefix="/api")
@bp.route("/capsul/create", methods=["POST"])
@account_required
def capsul_create():
email = session["account"]
from .console import _create,get_account_balance, get_payments, get_vms
vm_sizes = get_model().vm_sizes_dict()
operating_systems = get_model().operating_systems_dict()
public_keys_for_account = get_model().list_ssh_public_keys_for_account(session["account"])
account_balance = get_account_balance(get_vms(), get_payments(), datetime.datetime.utcnow())
capacity_avaliable = current_app.config["HUB_MODEL"].capacity_avaliable(512*1024*1024)
request.json['ssh_authorized_key_count'] = 1
id, errors = _create(
vm_sizes,
operating_systems,
public_keys_for_account,
request.json)
if id is not None:
return jsonify(
id=id,
)
return jsonify(errors=errors)

View File

@ -0,0 +1,2 @@
DROP TABLE api_keys;
UPDATE schemaversion SET version = 18;

View File

@ -0,0 +1,9 @@
CREATE TABLE api_tokens (
id SERIAL PRIMARY KEY,
email TEXT REFERENCES accounts(email) ON DELETE RESTRICT,
name TEXT NOT NULL,
created TIMESTAMP NOT NULL DEFAULT NOW(),
token TEXT NOT NULL
);
UPDATE schemaversion SET version = 19;

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.7 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.2 KiB

View File

@ -241,6 +241,7 @@ thead {
background: #bdc7b812; background: #bdc7b812;
} }
td, th { td, th {
padding: 0.1em 1em; padding: 0.1em 1em;
} }
table.small td, table.small th { table.small td, table.small th {

View File

@ -1,35 +0,0 @@
html {
color: #241e1e !important;
background-color: #bdc7b8 !important;
}
a {
color:#00517a !important;
text-shadow: 1px 1px 0px #eee !important;
}
a:hover, a:active, a:visited {
color: #323417 !important;
}
.nav-links a {
text-shadow: 2px 2px 0px #eee !important;
}
h1, h2, h3, h4, h5 {
text-shadow: 2px 2px 0px #eee;
}
main {
border: 1px dashed #241e1e !important;
}
input, select, textarea {
color: #241e1e !important;
}
th {
border-right: 1px solid #eee !important;
text-align: left !important;
}

View File

@ -46,9 +46,7 @@
<a href="/payment/stripe">Add funds with Credit/Debit (stripe)</a> <a href="/payment/stripe">Add funds with Credit/Debit (stripe)</a>
<ul><li>notice: stripe will load nonfree javascript </li></ul> <ul><li>notice: stripe will load nonfree javascript </li></ul>
</li> </li>
{% if btcpay_enabled %}
<li><a href="/payment/btcpay">Add funds with Bitcoin/Litecoin/Monero (btcpay)</a></li> <li><a href="/payment/btcpay">Add funds with Bitcoin/Litecoin/Monero (btcpay)</a></li>
{% endif %}
<li>Cash: email <a href="mailto:treasurer@cyberia.club">treasurer@cyberia.club</a></li> <li>Cash: email <a href="mailto:treasurer@cyberia.club">treasurer@cyberia.club</a></li>
</ul> </ul>

View File

@ -31,7 +31,7 @@
{% if session["account"] %} {% if session["account"] %}
<a href="/console">Capsuls</a> <a href="/console">Capsuls</a>
<a href="/console/ssh">SSH Public Keys</a> <a href="/console/keys">SSH &amp; API Keys</a>
<a href="/console/account-balance">Account Balance</a> <a href="/console/account-balance">Account Balance</a>
{% endif %} {% endif %}

View File

@ -101,7 +101,7 @@
</div> </div>
<div class="row justify-start"> <div class="row justify-start">
<label class="align" for="ssh_authorized_keys">SSH Authorized Keys</label> <label class="align" for="ssh_authorized_keys">SSH Authorized Keys</label>
<a id="ssh_authorized_keys" href="/console/ssh">{{ vm['ssh_authorized_keys'] }}</a> <a id="ssh_authorized_keys" href="/console/keys">{{ vm['ssh_authorized_keys'] }}</a>
</div> </div>
</div> </div>

View File

@ -31,7 +31,7 @@
<p>(At least one month of funding is required)</p> <p>(At least one month of funding is required)</p>
{% elif no_ssh_public_keys %} {% elif no_ssh_public_keys %}
<p>You don't have any ssh public keys yet.</p> <p>You don't have any ssh public keys yet.</p>
<p>You must <a href="/console/ssh">upload one</a> before you can create a Capsul.</p> <p>You must <a href="/console/keys">upload one</a> before you can create a Capsul.</p>
{% elif not capacity_avaliable %} {% elif not capacity_avaliable %}
<p>Host(s) at capacity. No capsuls can be created at this time. sorry. </p> <p>Host(s) at capacity. No capsuls can be created at this time. sorry. </p>
{% else %} {% else %}

View File

@ -23,7 +23,6 @@
How do I log in? How do I log in?
<p>ssh to the ip provided to you using the cyberian user.</p> <p>ssh to the ip provided to you using the cyberian user.</p>
<pre class='code'>$ ssh cyberian@1.2.3.4</pre> <pre class='code'>$ ssh cyberian@1.2.3.4</pre>
<p>For more information, see <a href="/about-ssh">Understanding the Secure Shell Protocol (SSH)</a>.</p>
</li> </li>
<li> <li>
How do I change to the root user? How do I change to the root user?

View File

@ -1,17 +1,18 @@
{% extends 'base.html' %} {% extends 'base.html' %}
{% block title %}SSH Public Keys{% endblock %} {% block title %}SSH &amp; API Keys{% endblock %}
{% block content %} {% block content %}
<div class="row third-margin"> <div class="row third-margin">
<h1>SSH PUBLIC KEYS</h1> <h1>SSH PUBLIC KEYS</h1>
</div> </div>
<div class="row third-margin"><div> <div class="row third-margin"><div>
{% if has_ssh_public_keys %} <hr/> {% endif %} {% if ssh_public_keys|length > 0 %} <hr/> {% endif %}
{% for ssh_public_key in ssh_public_keys %} {% for ssh_public_key in ssh_public_keys %}
<form method="post"> <form method="post">
<input type="hidden" name="method" value="DELETE"></input> <input type="hidden" name="method" value="DELETE"></input>
<input type="hidden" name="action" value="delete_ssh_key"></input>
<input type="hidden" name="name" value="{{ ssh_public_key['name'] }}"></input> <input type="hidden" name="name" value="{{ ssh_public_key['name'] }}"></input>
<input type="hidden" name="csrf-token" value="{{ csrf_token }}"/> <input type="hidden" name="csrf-token" value="{{ csrf_token }}"/>
<div class="row"> <div class="row">
@ -22,13 +23,14 @@
</form> </form>
{% endfor %} {% endfor %}
{% if has_ssh_public_keys %} <hr/> {% endif %} {% if ssh_public_keys|length > 0 %} <hr/> {% endif %}
<div class="third-margin"> <div class="third-margin">
<h1>UPLOAD A NEW SSH PUBLIC KEY</h1> <h1>UPLOAD A NEW SSH PUBLIC KEY</h1>
</div> </div>
<form method="post"> <form method="post">
<input type="hidden" name="method" value="POST"></input> <input type="hidden" name="method" value="POST"></input>
<input type="hidden" name="action" value="upload_ssh_key"></input>
<input type="hidden" name="csrf-token" value="{{ csrf_token }}"/> <input type="hidden" name="csrf-token" value="{{ csrf_token }}"/>
<div class="row justify-start"> <div class="row justify-start">
<label class="align" for="content">File Contents</label> <label class="align" for="content">File Contents</label>
@ -54,6 +56,51 @@
</div> </div>
</form> </form>
</div></div> </div></div>
<hr/>
<div class="row third-margin">
<h1>API KEYS</h1>
</div>
<div class="row third-margin"><div>
{% if generated_api_token %}
<hr/>
Generated key:
<span class="code">{{ generated_api_token }}</span>
{% endif %}
{% if api_tokens|length >0 %} <hr/>{% endif %}
{% for api_token in api_tokens %}
<form method="post">
<input type="hidden" name="method" value="DELETE"></input>
<input type="hidden" name="action" value="delete_api_token"></input>
<input type="hidden" name="id" value="{{ api_token['id'] }}"></input>
<input type="hidden" name="csrf-token" value="{{ csrf_token }}"/>
<div class="row">
<span class="code">{{ api_token['name'] }}</span>
created {{ api_token['created'].strftime("%b %d %Y") }}
<input type="submit" value="Delete">
</div>
</form>
{% endfor %}
{% if api_tokens|length >0 %} <hr/>{% endif %}
<div class="third-margin">
<h1>GENERATE A NEW API KEY</h1>
</div>
<form method="post">
<input type="hidden" name="method" value="POST"></input>
<input type="hidden" name="action" value="generate_api_token"></input>
<input type="hidden" name="csrf-token" value="{{ csrf_token }}"/>
<div class="smalltext">
<p>Generate a new API key, to integrate with other systems.</p>
</div>
<div class="row justify-start">
<label class="align" for="name">Key Name</label>
<input type="text" id="name" name="name"></input> (defaults to creation time)
</div>
<div class="row justify-end">
<input type="submit" value="Generate">
</div>
</form>
</div></div>
{% endblock %} {% endblock %}
{% block pagesource %}/templates/ssh-public-keys.html{% endblock %} {% block pagesource %}/templates/ssh-public-keys.html{% endblock %}

View File

@ -6,38 +6,22 @@
<div class="row third-margin"> <div class="row third-margin">
<h1>CAPSUL TYPES & PRICING</h1> <h1>CAPSUL TYPES & PRICING</h1>
</div> </div>
<div class="row half-margin">
<table>
<thead>
<tr>
<th>type</th>
<th>monthly*</th>
<th>cpus</th>
<th>mem</th>
<th>ssd</th>
<th>net</th>
</tr>
</thead>
<tbody>
{% for vm_size_key, vm_size in vm_sizes.items() %}
<tr>
<td>{{ vm_size_key }}</td>
<td>${{ vm_size['dollars_per_month'] }}</td>
<td>{{ vm_size['vcpus'] }}</td>
<td>{{ vm_size['memory_mb'] }}</td>
<td>25G</td>
<td>{{ vm_size['bandwidth_gb_per_month'] }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
<div class="row half-margin"> <div class="row half-margin">
<pre> <pre>
type monthly* cpus mem ssd net*
----- ------- ---- --- --- ---
f1-xs $5.00 1 512M 25G .5TB
f1-s $7.50 1 1024M 25G 1TB
f1-m $12.50 1 2048M 25G 2TB
f1-l $20.00 2 3072M 25G 3TB
f1-x $27.50 3 4096M 25G 4TB
f1-xx $50.00 4 8192M 25G 5TB
* net is calculated as a per-month average * net is calculated as a per-month average
* vms are billed for a minimum of 24 hours upon creation * vms are billed for a minimum of 24 hours upon creation
* all VMs come standard with one public IPv4 address * all VMs come standard with one public IPv4 address
SUPPORTED OPERATING SYSTEMS: SUPPORTED OPERATING SYSTEMS:

View File

View File

@ -0,0 +1,23 @@
from flask import url_for, session
from capsulflask.db import get_model
from capsulflask.tests_base import BaseTestCase
class LoginTests(BaseTestCase):
render_templates = False
def test_login_request(self):
with self.client as client:
response = client.get(url_for("auth.login"))
self.assert_200(response)
# FIXME test generated login link
def test_login_magiclink(self):
token, ignoreCaseMatches = get_model().login('test@example.com')
with self.client as client:
response = client.get(url_for("auth.magiclink", token=token))
self.assertRedirects(response, url_for("console.index"))
self.assertEqual(session['account'], 'test@example.com')

View File

@ -0,0 +1,170 @@
from unittest.mock import patch
from flask import url_for
from capsulflask.hub_model import MockHub
from capsulflask.db import get_model
from capsulflask.tests_base import BaseTestCase
class ConsoleTests(BaseTestCase):
capsul_data = {
"size": "f1-xs",
"os": "debian10",
"ssh_authorized_key_count": 1,
"ssh_key_0": "key"
}
ssh_key_data = {
"name": "key2",
"action": "upload_ssh_key",
"content": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDntq1t8Ddsa2q4p+PM7W4CLYYmxakokRRVLlf7AQlsTJFPsgBe9u0zuoOaKDMkBr0dlnuLm4Eub1Mj+BrdqAokto0YDiAnxUKRuYQKuHySKK8bLkisi2k47jGBDikx/jihgiuFTawo1mYsJJepC7PPwZGsoCImJEgq1L+ug0p3Zrj3QkUx4h25MpCSs2yvfgWjDyN8hEC76O42P+4ETezYrzrd1Kj26hdzHRnrxygvIUOtfau+5ydlaz8xQBEPrEY6/+pKDuwtXg1pBL7GmoUxBXVfHQSgq5s9jIJH+G0CR0ZoHMB25Ln4X/bsCQbLOu21+IGYKSDVM5TIMLtkKUkERQMVWvnpOp1LZKir4dC0m7SW74wpA8+2b1IsURIr9ARYGJpCEv1Q1Wz/X3yTf6Mfey7992MjUc9HcgjgU01/+kYomoXHprzolk+22Gjfgo3a4dRIoTY82GO8kkUKiaWHvDkkVURCY5dpteLA05sk3Z9aRMYsNXPLeOOPfzTlDA0="
}
def test_index(self):
self._login('test@example.com')
with self.client as client:
response = client.get(url_for("console.index"))
self.assert_200(response)
def test_create_loads(self):
self._login('test@example.com')
with self.client as client:
response = client.get(url_for("console.create"))
self.assert_200(response)
def test_create_fails_capacity(self):
with self.client as client:
client.get(url_for("console.create"))
csrf_token = self.get_context_variable('csrf_token')
data = self.capsul_data
data['csrf-token'] = csrf_token
# Override MockHub.capacity_avaliable to always return False
with patch.object(MockHub, 'capacity_avaliable', return_value=False) as mock_method:
self.app.config["HUB_MODEL"] = MockHub()
client.post(url_for("console.create"), data=data)
capacity_message = \
'\n host(s) at capacity. no capsuls can be created at this time. sorry. \n '
self.assert_message_flashed(capacity_message, category='message')
self.assertEqual(
len(get_model().list_vms_for_account('test@example.com')),
0
)
mock_method.assert_called_with(512 * 1024 * 1024)
def test_create_fails_invalid(self):
with self.client as client:
client.get(url_for("console.create"))
csrf_token = self.get_context_variable('csrf_token')
data = self.capsul_data
data['csrf-token'] = csrf_token
data['os'] = ''
client.post(url_for("console.create"), data=data)
self.assert_message_flashed(
'OS is required',
category='message'
)
self.assertEqual(
len(get_model().list_vms_for_account('test@example.com')),
0
)
def test_create_succeeds(self):
with self.client as client:
client.get(url_for("console.create"))
csrf_token = self.get_context_variable('csrf_token')
data = self.capsul_data
data['csrf-token'] = csrf_token
response = client.post(url_for("console.create"), data=data)
# FIXME: mock create doesn't create, see #83
# vms = get_model().list_vms_for_account('test@example.com')
# self.assertEqual(
# len(vms),
# 1
# )
#
# vm_id = vms[0].id
#
# self.assertRedirects(
# response,
# url_for("console.index") + f'?{vm_id}'
# )
def test_keys_loads(self):
self._login('test@example.com')
with self.client as client:
response = client.get(url_for("console.ssh_api_keys"))
self.assert_200(response)
keys = self.get_context_variable('ssh_public_keys')
self.assertEqual(keys[0]['name'], 'key')
def test_keys_add_ssh_fails_invalid(self):
self._login('test@example.com')
with self.client as client:
client.get(url_for("console.ssh_api_keys"))
csrf_token = self.get_context_variable('csrf_token')
data = self.ssh_key_data
data['csrf-token'] = csrf_token
data_invalid_content = data
data_invalid_content['content'] = 'foo'
client.post(
url_for("console.ssh_api_keys"),
data=data_invalid_content
)
self.assert_message_flashed(
'Content must match "^(ssh|ecdsa)-[0-9A-Za-z+/_=@:. -]+$"',
category='message'
)
data_missing_content = data
data_missing_content['content'] = ''
client.post(url_for("console.ssh_api_keys"), data=data_missing_content)
self.assert_message_flashed(
'Content is required', category='message'
)
def test_keys_add_ssh_fails_duplicate(self):
self._login('test@example.com')
with self.client as client:
client.get(url_for("console.ssh_api_keys"))
csrf_token = self.get_context_variable('csrf_token')
data = self.ssh_key_data
data['csrf-token'] = csrf_token
data['name'] = 'key'
client.post(url_for("console.ssh_api_keys"), data=data)
self.assert_message_flashed(
'A key with that name already exists',
category='message'
)
data = self.ssh_key_data
data['csrf-token'] = csrf_token
data['name'] = 'key'
client.post(url_for("console.ssh_api_keys"), data=data)
self.assert_message_flashed(
'A key with that name already exists',
category='message'
)
def setUp(self):
self._login('test@example.com')
get_model().create_ssh_public_key('test@example.com', 'key', 'foo')
def tearDown(self):
get_model().delete_ssh_public_key('test@example.com', 'key')

View File

@ -0,0 +1,14 @@
from capsulflask.tests_base import BaseTestCase
class LandingTests(BaseTestCase):
#: Do not render templates, we're only testing logic here.
render_templates = False
def test_landing(self):
pages = ['/', 'pricing', 'faq', 'about-ssh', 'changelog', 'support']
with self.client as client:
for page in pages:
response = client.get(page)
self.assert_200(response)

View File

@ -0,0 +1,45 @@
from base64 import b64encode
import requests
from flask import url_for
from capsulflask.db import get_model
from capsulflask.tests_base import BaseLiveServerTestCase
class PublicAPITests(BaseLiveServerTestCase):
def test_server_is_up_and_running(self):
response = requests.get(self.get_server_url())
self.assertEqual(response.status_code, 200)
def test_capsul_create_succeeds(self):
response = requests.post(
self.get_server_url() +
url_for('publicapi.capsul_create'),
headers={'Authorization': self.token},
json={
'size': 'f1-xs',
'os': 'openbsd68',
'ssh_key_0': 'key'
}
)
self.assertEqual(response.status_code, 200)
# FIXME: mock create doesn't create, see #83
# vms = get_model().list_vms_for_account('test@example.com')
#
# self.assertEqual(
# len(vms),
# 1
# )
def setUp(self):
get_model().create_ssh_public_key('test@example.com', 'key', 'foo')
self.token = b64encode(
get_model().generate_api_token('test@example.com', 'apikey').encode('utf-8')
).decode('utf-8')
def tearDown(self):
get_model().delete_ssh_public_key('test@example.com', 'key')
get_model().delete_api_token('test@example.com', 1)

35
capsulflask/tests_base.py Normal file
View File

@ -0,0 +1,35 @@
import os
from nanoid import generate
from flask_testing import TestCase, LiveServerTestCase
from capsulflask import create_app
from capsulflask.db import get_model
class BaseSharedTestCase(object):
def create_app(self):
# Use default connection paramaters
os.environ['POSTGRES_CONNECTION_PARAMETERS'] = "host=localhost port=5432 user=postgres password=dev dbname=capsulflask_test"
os.environ['TESTING'] = '1'
os.environ['SPOKE_MODEL'] = 'mock'
os.environ['HUB_MODEL'] = 'mock'
return create_app()
def setUp(self):
pass
def tearDown(self):
pass
class BaseTestCase(BaseSharedTestCase, TestCase):
def _login(self, user_email):
get_model().login(user_email)
with self.client.session_transaction() as session:
session['account'] = user_email
session['csrf-token'] = generate()
class BaseLiveServerTestCase(BaseSharedTestCase, LiveServerTestCase):
pass

View File

@ -1,94 +0,0 @@
{% extends 'base.html' %}
{% block title %}Account Balance{% endblock %}
{% block content %}
<div class="row third-margin">
<h1>Account Balance: ${{ account_balance }}</h1>
</div>
<div class="half-margin">
{% if has_vms and has_payments and warning_text != "" %}
<div class="row">
<pre class="wrap">{{ warning_text }}</pre>
</div>
{% endif %}
<div class="row">
{% if has_payments %}
<div>
<div class="row third-margin">
<h1>Payments</h1>
</div>
<table>
<thead>
<tr>
<th>amount</th>
<th>date</th>
</tr>
</thead>
<tbody>
{% for payment in payments %}
<tr>
<td class="{{ payment['class_name'] }}">${{ payment["dollars"] }}</td>
<td class="{{ payment['class_name'] }}">{{ payment["created"] }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
<ul>
<li>
<h1>PAYMENT OPTIONS</h1>
<ul>
<li>
<a href="/payment/stripe">Add funds with Credit/Debit (stripe)</a>
<ul><li>notice: stripe will load nonfree javascript </li></ul>
</li>
{% if btcpay_enabled %}
<li><a href="/payment/btcpay">Add funds with Bitcoin/Litecoin/Monero (btcpay)</a></li>
{% endif %}
</ul>
</li>
</ul>
</div>
{% if has_vms %}
<div class="row third-margin">
<h1>Capsuls Billed</h1>
</div>
<div class="row">
<table class="small">
<thead>
<tr>
<th>id</th>
<th>created</th>
<th>deleted</th>
<th>$/month</th>
<th>months</th>
<th>$ billed</th>
</tr>
</thead>
<tbody>
{% for vm in vms_billed %}
<tr>
<td>{{ vm["id"] }}</td>
<td>{{ vm["created"] }}</td>
<td>{{ vm["deleted"] }}</td>
<td>${{ vm["dollars_per_month"] }}</td>
<td>{{ vm["months"] }}</td>
<td>${{ vm["dollars"] }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% endif %}
</div>
{% endblock %}
{% block pagesource %}/templates/create-capsul.html{% endblock %}

View File

@ -1,60 +0,0 @@
<html lang="en">
<head>
<!-- Namecoin Address: N2aVL6pHtBp7EtNGb3jpsL2L2NyjBNbiB1 -->
<link href="{{ url_for('static', filename='favicon.yolocolo.ico') }}" rel="icon">
<title>{% block title %}{% endblock %}{% if self.title() %} - {% endif %}Capsul</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1.0">
<meta name="Description" content="Cyberia Capsul">
{% block head %}{% endblock %}
<link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">
<link rel="stylesheet" href="{{ url_for('static', filename='style.yolocolo.css') }}">
</head>
<body>
<nav>
<div class="row justify-space-between half-margin">
<div>
🦉 <a href="/"><b>YOLOCOLO</b></a>
</div>
<div>
&nbsp;
{% if session["account"] %}
{ {{ session["account"] }} <a href="{{ url_for('auth.logout') }}">Log Out</a> }
{% else %}
<a href="{{ url_for('auth.login') }}">Login</a>
{% endif %}
</div>
</div>
<div class="row justify-center half-margin wrap nav-links">
<a href="/pricing">Pricing</a>
<a href="/faq">FAQ</a>
{% if session["account"] %}
<a href="/console">Capsuls</a>
<a href="/console/ssh">SSH Public Keys</a>
<a href="/console/account-balance">Account Balance</a>
{% endif %}
<a href="/support">Support</a>
</div>
</nav>
{% for message in get_flashed_messages() %}
<div class="flash">{{ message }}</div>
{% endfor %}
{% block custom_flash %}{% endblock %}
<main>
{% block content %}{% endblock %}
</main>
{% block subcontent %}{% endblock %}
<footer>
This server runs <a
href="https://giit.cyberia.club/~forest/capsul-flask">capsul-flask</a> by
Cyberia Computer Club, available under the <a
href="https://creativecommons.org/licenses/by-sa/4.0/">Attribution-ShareAlike
4.0 International</a> licence.<br/><br/>
<a href="https://git.autonomic.zone/3wordchant/capsul-flask/src/branch/yolocolo/capsulflask{% block pagesource %}{% endblock %}">View page source</a>
</footer>
</body>
</html>

View File

@ -1,68 +0,0 @@
{% extends 'base.html' %}
{% block title %}Capsuls{% endblock %}
{% block custom_flash %}
{% if created %}
<div class="flash green">{{ created }} successfully created!</div>
{% endif %}
{% endblock %}
{% block content %}
<div class="row third-margin">
<h1>Capsuls</h1>
</div>
<div class="third-margin">
{% if has_vms %}
<div class="row third-margin justify-end">
<a href="/console/create">Create Capsul</a>
</div>
<div class="row">
<table>
<thead>
<tr>
<th class="heart-icon"></th>
<th>id</th>
<th>size</th>
<th>cpu</th>
<th>mem</th>
<th>ipv4</th>
<th>os</th>
<th>created</th>
</tr>
</thead>
<tbody>
{% for vm in vms %}
<tr>
{% if vm['state'] == 'starting' or vm['state'] == 'stopping' %}
<td class="capsul-status waiting-pulse"></td>
{% elif vm['state'] == 'crashed' or vm['state'] == 'blocked' or vm['state'] == 'stopped' %}
<td class="capsul-status red"></td>
{% elif vm['state'] == 'unknown' %}
<td class="capsul-status-questionmark">?</td>
{% else %}
<td class="capsul-status green"></td>
{% endif %}
<td><a class="no-shadow" href="/console/{{ vm['id'] }}">{{ vm["id"] }}</a></td>
<td>{{ vm["size"] }}</td>
<td class="metrics"><img src="/metrics/cpu/{{ vm['id'] }}/5m/s"/></td>
<td class="metrics"><img src="/metrics/memory/{{ vm['id'] }}/5m/s"/></td>
<td class="{{ vm['ipv4_status'] }}">{{ vm["ipv4"] }}</td>
<td>{{ vm["os"] }}</td>
<td>{{ vm["created"] }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<div class="row">
<p>You don't have any Capsuls running. <a href="/console/create">Create one</a> today!</p>
</div>
{% endif %}
</div>
{% endblock %}
{% block pagesource %}/templates/capsuls.html{% endblock %}

View File

@ -1,46 +0,0 @@
{% extends 'base.html' %}
{% block title %}FAQ{% endblock %}
{% block content %}
<div class="row full-margin"><h1>Frequently Asked Questions</h1></div>
{% endblock %}
{% block subcontent %}
<p>
<ul>
<li>
What is this?
<p>
This is a <strong>technical demo</strong> of <a
href="https://giit.cyberia.club/~forest/capsul-flask">Capsul</a>, for the
as-yet-untitled <a href="https://coops.tech">Cotech</a> server hosting
initiative, which you can <a
href="https://community.coops.tech/t/call-for-input-v2-co-op-vps-survey/2802/9">read
about on the Cotech forum</a>.
</p>
</li>
<li>
What do you mean, "technical demo"?
<p>No backups</p>
<p>No service level agreement</p>
<p>"Best effort" support</p>
</li>
<li>
Where can I get this, but, more reliable?
<p>Cyberia, the authors of this platform, run the canonical instance, <a
href="https://capsul.org">Capsul.org</a>, on hardware they own. Please
send them your money! (cash, crypto, or card accepted).</p>
</li>
<li>
How do I use this system?
<p>Please see <a href="https://capsul.org/faq">the official Capsul FAQ
page</a>.</p>
</li>
</ul>
</p>
{% endblock %}
{% block pagesource %}/templates/faq.html{% endblock %}

View File

@ -1,28 +0,0 @@
{% extends 'base.html' %}
{% block content %}
<h1>
<pre>
_ _
_ _ ___ | | ___ ___ ___ | | ___
| | | |/ _ \| |/ _ \ / __/ _ \| |/ _ \
| |_| | (_) | | (_) | (_| (_) | | (_) |
\__, |\___/|_|\___/ \___\___/|_|\___/
|___/
</pre>
<span>Co-operative hosting using <a href="https://cyberia.club">Cyberia</a>'s Capsul</span>
{% endblock %}
{% block subcontent %}
<p>
<ul>
<li>Sign up for an account!</li>
<li>Add some funds!</li>
<li>Create a VPS!</li>
<li>Give your feedback!</li>
</ul>
</p>
{% endblock %}
{% block pagesource %}/templates/index.html{% endblock %}

View File

@ -1,23 +0,0 @@
{% extends 'base.html' %}
{% block title %}Pricing{% endblock %}
{% block content %}
<div class="row third-margin">
<h1>CAPSUL TYPES & PRICING</h1>
</div>
<div class="row half-margin">
<p>
Rates for this service aren't set yet. You can see Cyberia's Capsul pricing
on <a href="https://capsul.org/pricing">their website</a>.
</p>
</div>
<div>
<pre>
SUPPORTED OPERATING SYSTEMS:
{% for os_id, os in operating_systems.items() %} - {{ os.description }}
{% endfor %}
</pre>
</div>
{% endblock %}

View File

@ -1,21 +0,0 @@
{% extends 'base.html' %}
{% block title %}Support{% endblock %}
{% block content %}
<div class="row half-margin">
<h1>SUPPORT</h1>
</div>
<div class="row half-margin">
<a href="mailto:yolocolo@doesthisthing.work?subject=Please%20help!">yolocolo@doesthisthing.work</a>
</div>
{% endblock %}
{% block subcontent %}
<p>
You can also find us on Matrix: <a
href="https://matrix.to/#/#untitled-hosting.public:autonomic.zone">#untitled-hosting.public:autonomic.zone</a>.
</p>
{% endblock %}
{% block pagesource %}/templates/support.html{% endblock %}

View File

@ -1,36 +0,0 @@
---
version: "3.8"
services:
app:
image: 3wordchant/capsul-flask:latest
build: .
volumes:
- "./:/app/code"
- "../tank:/tank"
# - "/var/run/libvirt/libvirt-sock:/var/run/libvirt/libvirt-sock"
depends_on:
- db
ports:
- "5000:5000"
environment:
- "POSTGRES_CONNECTION_PARAMETERS=host=db port=5432 user=capsul password=capsul dbname=capsul"
- SPOKE_MODEL=shell-scripts
#- FLASK_DEBUG=1
- BASE_URL=http://localhost:5000
- ADMIN_PANEL_ALLOW_EMAIL_ADDRESSES=3wc.capsul@doesthisthing.work
- VIRSH_DEFAULT_CONNECT_URI=qemu:///system
# The image uses gunicorn by default, let's override it with Flask's
# built-in development server
command: ["flask", "run", "-h", "0.0.0.0", "-p", "5000"]
db:
image: "postgres:9.6.5-alpine"
volumes:
- "postgres:/var/lib/postgresql/data"
environment:
POSTGRES_USER: capsul
POSTGRES_PASSWORD: capsul
POSTGRES_DB: capsul
volumes:
postgres:

View File

@ -1,30 +0,0 @@
# hub-and-spoke architecture
The "Hub" runs the web application and talks to the Postrges database, while the "Spoke"s are responsible for creating/managing virtual machines. One instance of the capsul-flask application can run in hub mode and spoke mode at the same time.
The Hub and the Spoke must be configured to communicate securely with each-other over HTTPS. They both have to be able to dial each-other directly. The URLs / auth tokens they use are configured both in the config file (`HUB_URL`, `SPOKE_HOST_ID`, `SPOKE_HOST_TOKEN` and `HUB_TOKEN`) and in the database (the `id`, `https_url`, and `token` columns in the `hosts` table).
![](images/hub-and-spoke1.png)
This diagram was created with https://app.diagrams.net/.
To edit it, download the <a download href="readme/hub-and-spoke.xml">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.
right now I have 2 types of operations, immediate mode and async.
both types of operations do assignment synchronously. so if the system cant assign the operation to one or more hosts (spokes),
or whatever the operation requires, then it will fail.
some operations tolerate partial failures, like, `capacity_avaliable` will succeed if at least one spoke succeeds.
for immediate mode requests (like `list`, `capacity_avaliable`, `destroy`), assignment and completion of the operation are the same thing.
for async ones, they can be assigned without knowing whether or not they succeeded (`create`).
![](images/hub-and-spoke2.png)
This diagram was created with https://app.diagrams.net/.
To edit it, download the <a download href="readme/hub-and-spoke.xml">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.
if you issue a create, and it technically could go to any number of hosts, but only one host responds, it will succeed
but if you issue a create and somehow 2 hosts both think they own that task, it will fail and throw a big error. cuz it expects exactly 1 to own the create task
currently its not set up to do any polling. its not really like a queue at all. It's all immediate for the most part

View File

@ -1,68 +0,0 @@
## <a name="BTCPAY_PRIVATE_KEY"></a>Setting up the BTCPAY_PRIVATE_KEY
Generate a private key and the accompanying bitpay SIN for the btcpay API client.
I used this code as an example: https://github.com/bitpay/bitpay-python/blob/master/bitpay/key_utils.py#L6
```
$ pipenv run python ./readme/generate_btcpay_keys.py
```
It should output something looking like this:
```
-----BEGIN EC PRIVATE KEY-----
EXAMPLEIArx/EXAMPLEKH23EXAMPLEsYXEXAMPLE5qdEXAMPLEcFHoAcEXAMPLEK
oUQDQgAEnWs47PT8+ihhzyvXX6/yYMAWWODluRTR2Ix6ZY7Z+MV7v0W1maJzqeqq
NQ+cpBvPDbyrDk9+Uf/sEaRCma094g==
-----END EC PRIVATE KEY-----
EXAMPLEwzAEXAMPLEEXAMPLEURD7EXAMPLE
```
In order to register the key with the btcpay server, you have to first generate a pairing token using the btcpay server interface.
This requires your btcpay server account to have access to the capsul store. Ask Cass about this.
Navigate to `Manage store: Access Tokens` at: `https://btcpay.cyberia.club/stores/<store-id>/Tokens`
![](images/btcpay_sin_pairing.jpg)
Finally, send an http request to the btcpay server to complete the pairing:
```
curl -H "Content-Type: application/json" https://btcpay.cyberia.club/tokens -d "{'id': 'EXAMPLEwzAEXAMPLEEXAMPLEURD7EXAMPLE', 'pairingCode': 'XXXXXXX'}"
```
It should respond with a token:
```
{"data":[{"policies":[],"pairingCode":"XXXXXXX","pairingExpiration":1589473817597,"dateCreated":1589472917597,"facade":"merchant","token":"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx","label":"capsulflask"}]}
```
And you should see the token in the btcpay server UI:
![](images/paired.jpg)
Now simply set your `BTCPAY_PRIVATE_KEY` variable in `.env`
NOTE: make sure to use single quotes and replace the new lines with \n.
```
BTCPAY_PRIVATE_KEY='-----BEGIN EC PRIVATE KEY-----\nEXAMPLEIArx/EXAMPLEKH23EXAMPLEsYXEXAMPLE5qdEXAMPLEcFHoAcEXAMPLEK\noUQDQgAEnWs47PT8+ihhzyvXX6/yYMAWWODluRTR2Ix6ZY7Z+MV7v0W1maJzqeqq\nNQ+cpBvPDbyrDk9+Uf/sEaRCma094g==\n-----END EC PRIVATE KEY-----'
```
-----
## <a name="testing"></a>testing cryptocurrency payments
I used litecoin to test cryptocurrency payments, because its the simplest & lowest fee cryptocurrency that BTCPay server supports. You can download the easy-to-use litecoin SPV wallet `electrum-ltc` from [github.com/pooler/electrum-ltc](https://github.com/pooler/electrum-ltc) or [electrum-ltc.org](https://electrum-ltc.org/), set up a wallet, and then either purchase some litecoin from an exchange, or [ask Forest for some litecoin](https://sequentialread.com/capsul-rollin-onwards-with-a-web-application/#sqr-comment-container) to use for testing.
## <a name="0_conf_diagram"></a>sequence diagram explaining how BTC payment process works (how we accept 0-confirmation transactions 😀)
![btcpayment_process](images/btcpayment_process.png)
This diagram was created with https://app.diagrams.net/.
To edit it, download the <a download href="readme/btcpayment_process.drawio">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 10 KiB

View File

@ -1,89 +0,0 @@
# Configuring Capsul-Flask
Create a `.env` file to set up the application configuration:
```
nano .env
```
You can enter any environment variables referenced in [`__init__.py`](../capsulflask/__init__.py) to this file.
For example you may enter your SMTP credentials like this:
```
MAIL_USERNAME=forest@nullhex.com
MAIL_DEFAULT_SENDER=forest@nullhex.com
MAIL_PASSWORD=**************
```
## <a name="example"></a>Example configuration from capsul.org (production):
```
#LOG_LEVEL=DEBUG
BASE_URL="https://capsul.org"
# hub url is used by the SPOKE_MODE to contact the hub. Since this server is the hub,
# this is fine. In fact it runs into problems (routing related?) when I set it to capsul.org.
# similarly the baikal "spoke" (set up in the hosts table in the db) has "http://localhost:5000" as the https_url
HUB_URL="http://localhost:5000"
HUB_MODE_ENABLED="t"
SPOKE_MODE_ENABLED="t"
HUB_MODEL="capsul-flask"
SPOKE_MODEL="shell-scripts"
SPOKE_HOST_ID="baikal"
SPOKE_HOST_TOKEN="<redacted>"
HUB_TOKEN="<redacted>"
# smtp.. see https://flask-mail.readthedocs.io/en/latest/#configuring-flask-mail
MAIL_SERVER="smtp.nullhex.com"
# MAIL_USE_SSL means SMTP with STARTTLS
MAIL_USE_SSL=true
# MAIL_USE_TLS means SMTP wrapped in TLS
MAIL_USE_TLS=false
MAIL_PORT="465"
MAIL_USERNAME="capsul@nullhex.com"
MAIL_PASSWORD="<redacted>"
MAIL_DEFAULT_SENDER="capsul@nullhex.com"
# stripe
STRIPE_SECRET_KEY="sk_live_<redacted>"
STRIPE_PUBLISHABLE_KEY="pk_live_tGDHY7kBwqC71b4F0N7LZdGl00GZOw0iNJ"
# internal
SECRET_KEY="<redacted>"
POSTGRES_CONNECTION_PARAMETERS="sslmode=verify-full sslrootcert=letsencrypt-root-ca.crt host=postgres.cyberia.club port=5432 ...<redacted>"
# btcpay server
BTCPAY_URL="https://beeteeceepae2.cyberia.club"
BTCPAY_PRIVATE_KEY='-----BEGIN EC PRIVATE KEY-----\n<redacted>\n-----END EC PRIVATE KEY-----'
```
## <a name="config_that_lives_in_db"></a>Configuration-type-stuff that lives in the database
- `hosts` table:
- `id` (corresponds to `SPOKE_HOST_ID` in the config)
- `https_url`
- `token` (corresponds to `SPOKE_HOST_TOKEN` in the config)
- `os_images` table:
- `id`
- `template_image_file_name`
- `description`
- `deprecated`
- `vm_sizes` table:
- `id`
- `dollars_per_month`
- `memory_mb`
- `vcpus`
- `bandwidth_gb_per_month`
## <a name="docker_secrets"></a>Loading variables from files (docker secrets)
To support [Docker Secrets](https://docs.docker.com/engine/swarm/secrets/), you can also load secret values from files for example, to load `MAIL_PASSWORD` from `/run/secrets/mail_password`, set
```sh
MAIL_PASSWORD_FILE=/run/secrets/mail_password
```

View File

@ -1,50 +0,0 @@
# capsul-flask's relationship to its Database Server
Capsul has a ["hub and spoke" architecture](./architecture.md). The "Hub" runs the web application and talks to the Postrges database, while the "Spoke"s are responsible for creating/managing virtual machines. One instance of the capsul-flask application can run in both hub mode and spoke mode at the same time, however there must only be one instance of the app running in "Hub" mode at any given time.
The Postgres connections parameters are [configurable](./configuration.md).
## <a name="schema_management"></a>Database schema management (schema versions)
capsul-flask has a concept of a schema version. When the application starts, it will query the database for a table named `schemaversion` that has one row and one column (`version`). If the `version` it finds is not equal to the `desiredSchemaVersion` variable set in `db.py`, it will run migration scripts from the `schema_migrations` folder one by one until the `schemaversion` table shows the correct version.
For example, the script named `02_up_xyz.sql` should contain code that migrates the database from schema version 1 to schema version 2. Likewise, the script `02_down_xyz.sql` should contain code that migrates from schema version 2 back to schema version 1.
**IMPORTANT: if you need to make changes to the schema, make a NEW schema version. DO NOT EDIT the existing schema versions.**
In general, for safety, schema version upgrades should not delete data. Schema version downgrades will simply throw an error and exit for now.
## <a name="manual_queries"></a>Running manual database queries
You can manually mess around with the database like this:
```
pipenv run flask cli sql -f test.sql
```
```
pipenv run flask cli sql -c 'SELECT * FROM vms'
```
This one selects the vms table with the column name header:
```
pipenv run flask cli sql -c "SELECT string_agg(column_name::text, ', ') from information_schema.columns WHERE table_name='vms'; SELECT * from vms"
```
How to modify a payment manually, like if you get a chargeback or to fix customer payment issues:
```
$ pipenv run flask cli sql -c "SELECT id, created, email, dollars, invalidated from payments"
1, 2020-05-05T00:00:00, forest.n.johnson@gmail.com, 20.00, FALSE
$ pipenv run flask cli sql -c "UPDATE payments SET invalidated = True WHERE id = 1"
1 rows affected.
$ pipenv run flask cli sql -c "SELECT id, created, email, dollars, invalidated from payments"
1, 2020-05-05T00:00:00, forest.n.johnson@gmail.com, 20.00, TRUE
```
## how to view the logs on the database server (legion.cyberia.club)
`sudo -u postgres pg_dump capsul-flask | gzip -9 > capsul-backup-2021-02-15.gz`

View File

@ -1,87 +0,0 @@
# Deploying Capsul on a server
Capsul has a ["hub and spoke" architecture](./architecture.md). The "Hub" runs the web application and talks to the Postrges database, while the "Spoke"s are responsible for creating/managing virtual machines. One instance of the capsul-flask application can run in both hub mode and spoke mode at the same time, however there must only be one instance of the app running in "Hub" mode at any given time.
## <a name="spoke_mode_prerequisites"></a>Installing prerequisites for Spoke Mode
On your spoke (see [Architecture](./architecture.md) You'll need `libvirtd`, `dnsmasq`, and `qemu-kvm`, plus a `/tank` diectory with some operating system images in it:
```
sudo apt install libvirt-daemon-system virtinst git dnsmasq qemu qemu-kvm
sudo mkdir -p /var/www /tank/{vm,img,config}
sudo mkdir -p /tank/img/debian/10
cd !$
sudo wget https://cloud.debian.org/images/cloud/buster/20201023-432/debian-10-genericcloud-amd64-20201023-432.qcow2 -O root.img.qcow2
```
TODO: network set-up
TODO: cyberia-cloudinit.yml
## Deploying capsul-flask
### <a name="deploy_manually"></a>Manually
Follow the [local set-up instructions](./local-set-up.md) on your server.
Make sure to set `BASE_URL` correctly, generate your own secret tokens, and
configure your own daemon management for the capsul-flask server (e.g. writing
init scripts, or SystemD unit files).
Use the suggested `gunicorn` command (with appropriately-set address and port),
instead of `flask run`, to launch the server.
For example, here is the SystemD service unit file we use in production for `capsul.org`:
```
[Unit]
Description=capsul-flask virtual machines as a service
After=network.target
[Service]
ExecStart=/usr/local/bin/pipenv run gunicorn --bind 127.0.0.1:5000 -k gevent --worker-connections 1000 app:app
Restart=on-failure
WorkingDirectory=/opt/capsul-flask
[Install]
WantedBy=multi-user.target
```
TODO: cron runner is required to run maintenance tasks for now, but in the future we want to build this into the python based task scheduler.
### <a name="coop_cloud_docker"></a> Using Co-op Cloud's vanilla Docker Swarm configuration
Download the Co-op Cloud swarm `compose.yml`:
```sh
wget https://git.autonomic.zone/coop-cloud/capsul/src/branch/main/compose.yml
```
Optionally, download add-on compose files for Stripe, BTCPay, and Spoke Mode:
```sh
wget https://git.autonomic.zone/coop-cloud/capsul/src/branch/main/compose.{stripe,btcpay,spoke}.yml
```
Then, create a `.env` file and configure appropriately -- you probably want to
define most settings in [the Co-op Cloud `.envrc.sample`
file](https://git.autonomic.zone/coop-cloud/capsul/src/branch/main/.envrc.sample).
Load the environment variables (using Python `direnv`, or a manual `set -a && source .env && set +a`), insert any necessary secrets, then run the deployment:
```sh
docker stack deploy -c compose.yml -c compose.stripe.yml your_capsul
```
(where you'd add an extra `-c compose.btcpay.yml` for each optional compose file
you want, and set `your_capsul` to the "stack name" you want).
TODO: cron runner
### <a name="coop_cloud_abra"></a> Using Co-op Cloud's `abra` deployment tool
Follow [the guide in the README for the Co-op Cloud capsul package](https://git.autonomic.zone/coop-cloud/capsul/).
### Using docker-compose
TODO

View File

@ -1,68 +0,0 @@
# How to run Capsul locally
## <a name="manually"></a>Manually
Ensure you have the pre-requisites for the psycopg2 Postgres database adapter package:
```sh
sudo apt install python3-dev libpq-dev
pg_config --version
```
Ensure you have the wonderful `pipenv` python package management and virtual environment cli:
```sh
sudo apt install pipenv
```
Create python virtual environment and install packages:
```sh
pipenv install
```
Run an instance of Postgres (I used docker for this, you can use whatever you want, point is its listening on `localhost:5432`):
```sh
docker run --rm -it -e POSTGRES_PASSWORD=dev -p 5432:5432 postgres
```
Run the app
```sh
pipenv run flask run
```
or, using Gunicorn:
```sh
pipenv run gunicorn --bind 127.0.0.1:5000 -k gevent --worker-connections 1000 app:app
```
Note that by default when running locally, the `SPOKE_MODEL` is set to `mock`, meaning that it won't actually try to spawn vms.
## Crediting your account
Once you log in for the first time, you will want to give yourself some free capsulbux so you can create fake capsuls for testing.
```sh
pipenv run flask cli sql -c "INSERT INTO payments (email, dollars) VALUES ('<your email address here>', 20.00)"
```
## Running scheduled tasks:
```sh
pipenv run flask cli cron-task
```
## <a name="docker_compose"></a>Run locally with docker-compose
If you have Docker and Docker-Compose installed, you can use the
`3wordchant/capsul-flask` Docker image to launch capsul-flask, and a Postgres
database server, for you:
```sh
docker-compose up
```
`capsul-flask` will read settings from your `.env` file as usual; you can set any of the options mentioned in the [configuration documentation](./configuration.md).

View File

Before

Width:  |  Height:  |  Size: 35 KiB

After

Width:  |  Height:  |  Size: 35 KiB

View File

Before

Width:  |  Height:  |  Size: 190 KiB

After

Width:  |  Height:  |  Size: 190 KiB

View File

Before

Width:  |  Height:  |  Size: 41 KiB

After

Width:  |  Height:  |  Size: 41 KiB

View File

Before

Width:  |  Height:  |  Size: 49 KiB

After

Width:  |  Height:  |  Size: 49 KiB

View File

Before

Width:  |  Height:  |  Size: 22 KiB

After

Width:  |  Height:  |  Size: 22 KiB