Merge branch 'develop'

3a3049d2 · dmorley · 365a3644 · 8f8aeb32 · 3a3049d2 · 3a3049d2
Commit 3a3049d2 authored Sep 03, 2018 by dmorley
46 changed files
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
+# Changelog
+The format is based on [Keep a Changelog] and this project adheres to [Semantic Versioning].
+## [2.4.0] - 2018-09-03
+:exclamation: DB migrations required! (see [SQL migration script][2.4.0-sql-migration])  
+:exclimation: config.php rewritten - requires migration    
+### Added
+- Added bootstrapping to simplify initialisation of config and database
+- Config syntax has changed to array style (#155)
+- Added `pghost` config to set database port
+- Added `CONTRIBUTING.md`
+- Podmin can choose at what fail score to send the notice out
+- Podmin email shares details on why pod is failing
+- Only retrieve location data for remote servers / IPs
+- Fontawesome for icons
+### Changed
+- Introduce proper changelog format (#189)
+- Moved DB migration scripts into `db` folder
+- Use Curl for all http calls
+- Use filter dropdowns for pre-defined columns
+- Use pretty URLs (see nginx.example)
+- Open pod URLs in a new tab
+- Use detectlanguage.com API for language guess
+- Only use JSON data, ignore HTML when returned
+- Allow curl redirect on home page check
+- Score now goes to -5000 before a pod is removed so dead pods get checked a while then removed for good
+- Move functions to dedicated file to allow reuse
+- Backup script rewrite
+- Store services as json array  
+- Paging can be changed to some or all on advanced view
+### Deprecated
+### Removed
+### Fixed
+- Notify podmins just once at 50 when pod failing (#186)
+- Add missing meta and PHP module requirements to `composer.json`
+### Security
+## [2.3.1] - 2018-08-05
+### Added
+- Podmins can link directly to their pod via `https://podupti.me/domain.name` for stats and to allow users to rate easier
+- Wizard to help you filter the columns to what you need (#145)
+- Cookie used to remember last pod you clicked
+### Changed
+- Now one table with a basic default view you can customize (#171)
+- Switch to a library for country to lat long lookup
+- Switch GeoIP from built in PHP to library and use newer Maxmind database file
+## [2.3.0] - 2018-07-19
+:exclamation: DB migrations required! (see [SQL migration script][2.2.0-sql-migration])
+### Added
+- Language is detected based on your homepage, edit your homepage to non-en if that is what you use
+- Add development and release dates to `masterversions` table (#143)
+- Store full country name, store days monitored each pod (#150)
+- Store detectedlanguage (#144)
+- Show version and update in full view cleaner (#143)
+- Filter and search on the columns of data (#147)
+- Paginate the results so they fit per page (#147)
+### Changed
+- Podmins can no longer access `db/pull.php` to test their pod, they can however get to a debug screen from the edit pod area
+- Edit will send to email on file and be less delay, runner of site does not really have any way to verify email address
+- Default new pods to `UP` to be checked
+- Use the git API for release versions, check development releases on pods (#143)
+- Move from [bower to yarn](https://bower.io/blog/2017/how-to-migrate-away-from-bower/) for packages
+- Move to PHP 7.2 with strict typing
+- Move to [Eslint compliance](https://eslint.org/docs/rules/)
+- Move to [PSR-2 compliance](https://www.php-fig.org/psr/psr-2/)
+- NOTE `config.php.example` change to full paths for 2 items!
+- Show time as human readable everywhere (#150)
+### Removed
+- Unused `hidden` and `secure` columns (#140, #141)
+### Fixed
+- Rename table `rating_comments` to `ratingcomments` for redbean support (#146)
+### Security
+- Forbid access to files that should be CLI only (#152)
+## [2.2.0] - 2018-05-12
+:exclamation: DB migrations required! (see [SQL migration script][2.2.0-sql-migration])
+### Added
+- Podmins can now pause/unpause or delete from podmin area
+- Graph on user growth on the network
+- Add monthly stats table
+### Changed
+- `go.php` auto select picks a more stable pod than before
+- Make map prettier
+- Use lines on tables to make them more readable
+- Don't delete dead pods, keep them and data for history hide them for users
+- Put daily tasks in the `pull.sh` and run each day
+- Update status to 1-5 rather than text
+### Fixed
+- Fix ipv6
+[2.4.0-sql-migration]: https://git.feneas.org/diasporg/Poduptime/blob/master/db/migrations/2.3.0-2.4.0.sql
+[2.3.0-sql-migration]: https://git.feneas.org/diasporg/Poduptime/blob/master/db/migrations/2.2.0-2.3.0.sql
+[2.2.0-sql-migration]: https://git.feneas.org/diasporg/Poduptime/blob/master/db/migrations/2.1.4-2.2.0.sql
+[Unreleased]: https://git.feneas.org/diasporg/Poduptime/compare/master...develop
+[2.3.1]: https://git.feneas.org/diasporg/Poduptime/compare/2.3.0...2.3.1
+[2.3.0]: https://git.feneas.org/diasporg/Poduptime/compare/v2.2.0...2.3.0
+[2.2.0]: https://git.feneas.org/diasporg/Poduptime/compare/2.1.3...v2.2.0
+[Keep a Changelog]: https://keepachangelog.com/
+[Semantic Versioning]: https://semver.org/
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
+# How To Contribute
+## Workflow
+* Fork Git Repo https://git.feneas.org/diasporg/Poduptime
+* Pull Git
+* Create a topic branch to work from
+* Commit and push your branch up
+* Create a Merge Request to the develop branch
+## Guidelines
+* Note your changes in [`CHANGELOG.md`] following https://keepachangelog.com 
+* Create any necessary DB migration script in [`db/migrations`] and note them in the changelog.
+* Update [`README.md`] with needed changes
+* Update the [Wiki] with API changes
+* Write your tests and validate them before you do your MR
+## Coding Style
+* PHP follows [PSR-2]
+* CSS follows [CSSLint]
+* JS follows [ESLint]
+[`README.md`]: https://git.feneas.org/diasporg/Poduptime/blob/master/README.md
+[`CHANGELOG.md`]: https://git.feneas.org/diasporg/Poduptime/blob/master/CHANGELOG.md
+[`db/migrations`]: https://git.feneas.org/diasporg/Poduptime/tree/master/db/migrations
+[Wiki]: https://git.feneas.org/diasporg/Poduptime/wikis/home
+[PSR-2]: https://www.php-fig.org/psr/psr-2/
+[CSSLint]: https://github.com/CSSLint/csslint
+[ESLint]: https://eslint.org/docs/rules/
--- a/README.md
+++ b/README.md
@@ -8,9 +8,8 @@ Environmental items you need (debian based system assumed):
 OS Dependencies:
 ```
-php7.2 php7.2-curl php7.2-pgsql php-geoip php7.2-cli php7.2-common php7.2-bcmath php7.2-json php7.2-readline php7.2-mbstring php7.2-xml php-cgi git curl postgresql postgresql-contrib dnsutils bind9 npm nodejs composer
+php7.2 php7.2-curl php7.2-pgsql php7.2-bcmath php-geoip php7.2-cli php7.2-common php7.2-bcmath php7.2-json php7.2-readline php7.2-mbstring php7.2-xml php-cgi git curl postgresql postgresql-contrib dnsutils bind9 npm nodejs composer
 ```
-GeoIP needs setup normally with a dat file
 Yarn is a separate install: https://yarnpkg.com
@@ -54,13 +53,14 @@ run `db/pull.sh debug` to debug output
 run `db/pull.sh sqldebug` to debug sql  
 run `db/pull.sh develop` to run without email alerts to end users  
 run `db/pull.sh Check_System_Deleted` to re-check system deleted pods as needed  
+These commands can be combined  
 # To Upgrade:
 ```
 git pull
 yarn install
 composer install
-psql -U podupuser podupdb < db/migrationXXX.sql (see db/version.md for proper migration versions)
+psql -U podupuser podupdb < db/migrations/xxx.sql (see db/migrations/README.md for proper migration versions)
 ```
 # Status

--- a/api.php
+++ b/api.php
@@ -15,15 +15,7 @@ $_format   = $_GET['format'] ?? '';
 $_method   = $_GET['method'] ?? '';
 $_callback = $_GET['callback'] ?? '';
-require_once __DIR__ . '/vendor/autoload.php';
+require_once __DIR__ . '/boot.php';
-require_once __DIR__ . '/config.php';
-define('PODUPTIME', microtime(true));
-// Set up global DB connection.
-R::setup("pgsql:host={$pghost};dbname={$pgdb}", $pguser, $pgpass, true);
-R::testConnection() || die('Error in DB connection');
-R::usePartialBeans(true);
 if ($_format === 'georss') {
    echo <<<EOF

--- a/boot.php
+++ b/boot.php
+<?php
+/**
+ * Boot the application, loading config and database connection.
+ */
+declare(strict_types=1);
+use DetectLanguage\DetectLanguage;
+use RedBeanPHP\R;
+define('PODUPTIME', microtime(true));
+require_once __DIR__ . '/vendor/autoload.php';
+// Initialise language detection.
+DetectLanguage::setApiKey(c('dlkey'));
+DetectLanguage::setSecure(true);
+// Set up global DB connection.
+R::setup(
+    sprintf(
+        'pgsql:host=%s;port=%d;dbname=%s',
+        c('pghost'),
+        c('pgport', 5432),
+        c('pgdb')
+    ),
+    c('pguser'),
+    c('pgpass'),
+    true
+);
+R::testConnection() || die('Error in DB connection');
+R::usePartialBeans(true);
--- a/composer.json
+++ b/composer.json
 {
+    "name": "diasporg/poduptime",
+    "description": "Poduptime is software to get live stats and data on listed Diaspora Pods.",
+    "license": "AGPL-3.0-or-later",
    "config": {
        "platform": {
            "php": "7.2"
@@ -6,22 +9,29 @@
    },
    "require": {
        "php": "^7.2",
-        "gabordemooij/redbean": "^5.0",
+        "ext-curl": "*",
-        "nesbot/carbon": "^1.31",
+        "ext-dom": "*",
+        "ext-json": "*",
+        "ext-libxml": "*",
+        "ext-bcmath": "*",
+        "gabordemooij/redbean": "^5.1",
+        "nesbot/carbon": "^1.33",
        "commerceguys/enum": "^1.0",
        "noplanman/xec": "0.1.0",
-        "jaybizzle/crawler-detect": "1.*",
+        "jaybizzle/crawler-detect": "^1.2",
-        "patrickschur/language-detection": "^3.3",
        "geoip2/geoip2": "^2.9",
-        "maxmind-db/reader": "~1.0",
+        "maxmind-db/reader": "^1.3",
        "matriphe/iso-639": "^1.2",
-        "rinvex/country": "^3.1"
+        "rinvex/country": "^3.1",
+        "detectlanguage/detectlanguage": "^2.2",
+        "longman/ip-tools": "^1.2"
    },
    "require-dev": {
        "squizlabs/php_codesniffer": "^3.3"
    },
    "autoload": {
-        "classmap": ["lib"]
+        "classmap": ["lib"],
+        "files": ["lib/functions.php"]
    },
    "scripts": {
        "check-code": [

--- a/composer.lock
+++ b/composer.lock
--- a/config.php.example
+++ b/config.php.example
@@ -4,38 +4,46 @@
 * Config for Poduptime.
 */
-//backup directory - full dir path
+return [
-$backup_dir = __DIR__ . '/backup';
+    //backup directory - full dir path
+    'backup_dir'  => __DIR__ . '/backup',
-//log directory - full dir path
+    //log directory - full dir path
-$log_dir = __DIR__ . '/log';
+    'log_dir'     => __DIR__ . '/log',
-//location of pg dump - full dir path
+    //location of pg dump - full dir path
-$pg_dump_dir = '/usr/bin';
+    'pg_dump_dir' => '/usr/bin',
-//db host
+    //db host
-$pghost = 'localhost';
+    'pghost'      => 'localhost',
-//db username
+    //db port
-$pguser = '';
+    'pgport'      => 5432,
-//db password
+    //db username
-$pgpass = '';
+    'pguser'      => '',
-//db name
+    //db password
-$pgdb = '';
+    'pgpass'      => '',
-//admin email for forms
+    //db name
-$adminemail = '';
+    'pgdb'        => '',
-//DNS server for dnssec testing. 1.1.1.1 tests the best
+    //admin email for forms
-$dnsserver = '';
+    'adminemail'  => '',
-//CA for curl to use - full file path (pull.sh will update this monthly)
+    //DNS server for dnssec testing. 1.1.1.1 tests the best
-$cafullpath = '';
+    'dnsserver'   => '',
-//Mapbox.com API key. https://www.mapbox.com/help/how-access-tokens-work/
+    //CA for curl to use - full file path (pull.sh will update this monthly)
-$mapboxkey = '';
+    'cafullpath'  => '',
-//Geolite2-city database file in mmdb format - full file path (pull.sh will update this monthly)
+    //Mapbox.com API key. https://www.mapbox.com/help/how-access-tokens-work/
-$geoip2db = '';
+    'mapboxkey'   => '',
+    //Geolite2-city database file in mmdb format - full file path (pull.sh will update this monthly)
+    'geoip2db'    => '',
+    //detectlanguage.com api key
+    'dlkey'       => '',
+];
--- a/css/poduptime.css
+++ b/css/poduptime.css
+@import url("/node_modules/typeface-roboto/index.css");
+body {
+    font-family: 'Roboto', sans-serif;
+    font-weight: 400;
+    --blue: #303030;
+    --grey: #4e89a4;
+    --green: #6B8E23;
+}
+.bg-blue {
+    background-color: var(--blue) !important;
+}
+.blue {
+    color: var(--blue);
+}
+.bg-green {
+    background-color: var(--green);
+}
+.green {
+    color: var(--green);
+}
+.bg-grey {
+    background-color: var(--grey);
+}
+.grey {
+    color: var(--grey);
+}
+a {
+    color: var(--blue);
+}
+.tablesorter-header {
+    background-color: var(--grey) !important;
+}
 .main {
    padding: 5px;
 }
@@ -25,35 +67,8 @@
 }
 .smlogo {
-    width: 16px;
-    height: 16px;
-    background: url('/images/smlogo.png') 0 0;
-    display: inline-block;
-    margin: 0;
-}
-.smlogo-twitter {
-    background-position: 0 -776px;
-}
-.smlogo-facebook {
-    background-position: 0 -824px;
-}
-.smlogo-tumblr {
-    background-position: 0 -792px;
-}
-.smlogo-wordpress {
-    background-position: 0 -656px;
-}
-.smlogo-xmpp {
    background: none;
-}
+    display: inline-block;
-.smlogo-xmpp img {
-    vertical-align: unset;
 }
 #map {

--- a/db/add.php
+++ b/db/add.php
@@ -8,9 +8,9 @@ declare(strict_types=1);
 use RedBeanPHP\R;
-require_once __DIR__ . '/../vendor/autoload.php';
+require_once __DIR__ . '/../boot.php';
-require_once __DIR__ . '/../config.php';
+<<<<<<< HEAD
 define('PODUPTIME', microtime(true));
@@ -23,11 +23,15 @@ R::usePartialBeans(true);
 if (!($_domain = $_GET['domain'] ?? null)) {
    die('no pod domain given');
 }
+=======
+($_domain = $_GET['domain'] ?? null) || die('no pod domain given');
+>>>>>>> develop
 // Other parameters.
-$_email            = $_GET['email'] ?? '';
+$_email                  = $_GET['email'] ?? '';
-$_podmin_statement = $_GET['podmin_statement'] ?? '';
+$_podmin_statement       = $_GET['podmin_statement'] ?? '';
-$_podmin_notify    = $_GET['podmin_notify'] ?? 0;
+$_podmin_notify          = $_GET['podmin_notify'] ?? 0;
+$_podmin_notify_level    = $_GET['podmin_notify_level'] ?? 50;
 $_domain = strtolower($_domain);
 if (!filter_var(gethostbyname($_domain), FILTER_VALIDATE_IP)) {
@@ -43,6 +47,7 @@ try {
    die('Error in SQL query: ' . $e->getMessage());
 }
+$stop = false;
 foreach ($pods as $pod) {
    if ($pod['domain'] === $_domain) {
        if ($pod['email']) {
@@ -69,6 +74,7 @@ foreach ($pods as $pod) {
        }
        echo <<<EOF
+<<<<<<< HEAD
      <script>
      window.onload = function() {
          $("#weight").ionRangeSlider();
@@ -86,11 +92,35 @@ foreach ($pods as $pod) {
 EOF;
        $stop = 1;
+=======
+<script>
+    window.onload = function() {
+        $("#weight").ionRangeSlider();
+        $("#podmin_notify_level").ionRangeSlider();
+    }
+</script>
+<form action="/">
+    <input type="hidden" name="edit">
+    <input type="hidden" name="domain" value="{$_domain}">
+    <input type="hidden" name="token" value="{$uuid}">
+    <label>Email <input type="text" size="20" name="email"></label><br>
+    <label>Podmin Statement (You can include links to your terms and policies and information about your pod you wish to share with users.) <br><textarea cols="100" rows="7" name="podmin_statement"></textarea></label><br>
+    <label>Weight <input type="text" data-min="0" data-step="1" data-max="10" name="weight" id="weight"> This lets you weight your pod lower on the list if you have too much traffic coming in, 10 is the norm use lower to move down the list.</label><br>
+    <label>Notify when your score falls to <input type="text" data-min="1" data-step="1" data-max="99" data-from="50" name="podmin_notify_level" id="podmin_notify_level"></label><br>
+    <input type="submit" name="action" value="save">
+</form>
+EOF;
+        $stop = true;
+>>>>>>> develop
    }
 }
 if (!$stop) {
+<<<<<<< HEAD
+=======
+>>>>>>> develop
    $link = 'https://' . $_domain . '/nodeinfo/1.0';
    if ($infos = file_get_contents('https://' . $_domain . '/.well-known/nodeinfo')) {
        $info = json_decode($infos, true);
@@ -111,6 +141,7 @@ if (!$stop) {
        echo 'Your pod has ssl and is valid<br>';
        $publickey = md5(uniqid($_domain, true));
+<<<<<<< HEAD
        try {
            $p = R::dispense('pods');
@@ -130,6 +161,28 @@ if (!$stop) {
            $subject = 'New pod added to ' . $_SERVER['HTTP_HOST'];
            $headers = ['From: ' . $_email, 'Reply-To: ' . $_email, 'Cc: ' . $_email];
+=======
+        try {
+            $p                           = R::dispense('pods');
+            $p['domain']                 = $_domain;
+            $p['email']                  = $_email;
+            $p['podmin_statement']       = $_podmin_statement;
+            $p['podmin_notify']          = $_podmin_notify;
+            $p['podmin_notify_level']    = $_podmin_notify_level;
+            $p['publickey']              = $publickey;
+            R::store($p);
+        } catch (\RedBeanPHP\RedException $e) {
+            die('Error in SQL query: ' . $e->getMessage());
+        }
+        if ($_email) {
+            $to      = c('adminemail');
+            $subject = 'New pod added to ' . $_SERVER['HTTP_HOST'];
+            $headers = ['From: ' . $_email, 'Reply-To: ' . $_email, 'Cc: ' . $_email];
+>>>>>>> develop
            $message_lines = [
                'https://' . $_SERVER['HTTP_HOST'],
                'Your pod ' . $_domain . ' will not show up right away, as it needs to pass a few checks first.',
@@ -144,4 +197,8 @@ if (!$stop) {
    } else {
        echo 'Could not validate your pod, check your setup!<br>Take a look at <a href="' . $link . '">your /nodeinfo</a>';
    }
-}
+<<<<<<< HEAD
\ No newline at end of file
+}
+=======
+}
+>>>>>>> develop
--- a/db/api-more.php
+++ b/db/api-more.php
@@ -15,15 +15,7 @@ use RedBeanPHP\R;
 // Other parameters.
 $_format = $_GET['format'] ?? '';
-require_once __DIR__ . '/../vendor/autoload.php';
+require_once __DIR__ . '/../boot.php';
-require_once __DIR__ . '/../config.php';
-define('PODUPTIME', microtime(true));
-// Set up global DB connection.
-R::setup("pgsql:host={$pghost};dbname={$pgdb}", $pguser, $pgpass, true);
-R::testConnection() || die('Error in DB connection');