Embeddable Discussions Ivelin Elenchev

Project Goal Provide easy to configure discussions using <iframe> Advanced configurable discussions using a web based API

Forums

Website Comments

Ecommerce reviews

System Level Requirements The system should be able to recognize discussion hosting websites and applications and return the requested information The system should be able to filter malicious requests via user activity, browser signature and IP The system should be able to monitor its resource usage and automatically reconfigure the storage and application cluster as needed

Moderator Level Requirements Administrators should be able to specify the discussion type (hierarchical, rating, review) Administrators should be able to specify custom discussion templates Administrators should be able to moderate discussions and assign privileges to users Administrators should be able to view usage statistics

User Level Requirements Users should be able to comment and reply to others Users should be able to log in from third party services via authentication tokens Users should be able to rate and flag other comments Users should be able to receive push & email notifications in special events (reply, rating, achievment)

Requirements Administrators should be able to specify the discussion type (hyrarchial, rating, review) Users should be able to comment and reply to others Users should be able to log in from third party services via authentication tokens Users should be able to rate and flag other comments Administrators should be able to specify custom discussion templates

Performance Requirements The system architecture should allow for cheap horizontal scaling The system should be able to function under high levels of IO (IO bound not CPU bound) Crashes in parts of the database or application clusters should not affect the rest of the system

Roadmap ✅ Storage “schema” + cluster configuration ✅ Storage caching ✅ User API Administrator API System API API Cluster configuration + monitoring Application Caching

Storage Postgres pros: battle tested, consistency (transactions) cons: partitioning relies on a master node, querying hirarchial comment data is slow (at least 2 joins) Cassanda: pros: reliable scaling, fault tolerant, performance under high IO cons: query language

Storage Caching Redis queries to Cassandra for the last minute are storedd in Redis. Cassandra is hit only if the Redis cache is too old, or does not exist. Non-priority users always get the Redis cache

Application Python pros: asynchronous web frameworks (Tornado), community cons: heavy system threading model Erlang / Elixir pros: asynchronous web framework (Phoenix), light VM threads, application cluster support cons: community

Webserver Architecture Nginx reverse proxy, static file caching, access limitation Cowboy Erlang webserver dealing with WSGI requests Varnish Cache API request caching

Questions?