2018-01-13

sph-dg network distribution conception

goal: sph-dg being capable of automatically utilising the combined storage space and processing power of multiple hosts

features

redundancy

synchronisation

automatic

setup

install software package, add address of at least one other host, start network daemon application

at

network-information

hosts, signals

status

insert-to

delete-to

modify-at

modify-from

unavailable

unreliable

btree

accepts-write response-time host-id status time origin-ids data-ranges direction free-space-percent cpu-rating

cache

origin-id response-time -> host-id

data-range-id start end -> host-id

update

overwrite with newer entries

add missing entries sorted by response-time limited

commit log

log write in same transaction as other data write

store commit-log reset mtime and log-id

timestamp commit-id -> change-type element-id data
mtime -> last-commit-id

change-type: 00/create 01/delete 10/update

data

origin identifier per host

origin identifier sub-range

allocate sub-range to acquired nodes

relations, nodes and records include origin id

key-value and indexes distribute by range

system information

list of all table/key-value spaces

signal

need node in mode x

from

get network-information

to

add origin/range

get network-information

get commits

set unavailable

retry count and timeout

node mode changes after coordination

set reavailable

redecide use

set unreliable

reliability-score since-time

acquisition

distribute by direction and magnitude

decide use

distribute

at write request is too late

if space usage reached

assign origin id

limit number of origins a read-only host can receive

decide by space usage of origin

replicants can bundle origins

replicants can become origins if necessary

redundancy

deduced from network-information


tags: computer sph-dg database design network distribution conception scaling