added entries

.gitignore

@@ -1,3 +1,4 @@
 /.idea
 /tools/archive/**
+/tools/conversion/**
 __pycache__

README.md

@@ -3,14 +3,14 @@
 [Dynamic HTML table](https://trilarion.github.io/opensourcegames/) of the entries / Development [Blog](https://trilarion.blogspot.com/search/label/osgames)
 
 [comment]: # (start of autogenerated content, do not edit)
-403 entries
+406 entries
-- **[Action/Arcade](games/action/_toc.md)** (18)
+- **[Action/Arcade](games/action/_toc.md)** (20)
 - **[Adventure & Visual novels](games/adventure/_toc.md)** (15)
 - **[Engines & Frameworks](games/framework/_toc.md)** (52)
 - **[Libraries](games/library/_toc.md)** (22)
 - **[Role Playing](games/rpg/_toc.md)** (105)
 - **[Simulation](games/simulation/_toc.md)** (15)
-- **[Strategy](games/strategy/_toc.md)** (164)
+- **[Strategy](games/strategy/_toc.md)** (165)
 - **[Tools & Manuals](games/misc/_toc.md)** (12)
 
 [comment]: # (end of autogenerated content)

docs/data.json

@@ -387,6 +387,15 @@
    "",
    "<a href=\"https://github.com/Trilarion/boson\">Source</a> - C++ - GPL-2.0"
   ],
+  [
+   "Boulder Dash (<a href=\"https://github.com/valeriansaliou/boulder-dash\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/action/boulder_dash.md\">entry</a>)",
+   "Boulder Dash game remake, done in Java.",
+   "<a href=\"https://github.com/valeriansaliou/boulder-dash/releases\">Link</a>",
+   "Action/Arcade",
+   "mature / inactive since 2015",
+   "",
+   "<a href=\"https://github.com/valeriansaliou/boulder-dash\">Source</a> - Java - MIT"
+  ],
   [
    "BrowserQuest (<a href=\"http://browserquest.mozilla.org/\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/rpg/browserquest.md\">entry</a>)",
    "A Massively Multiplayer Adventure.",
@@ -655,7 +664,7 @@
    "Strategy",
    "mature / inactive since 2016",
    "MMO",
-   "<a href=\"https://svn.code.sf.net/p/darkdestiny/code/\">Source</a> - Java"
+   "<a href=\"https://svn.code.sf.net/p/darkdestiny/code/\">Source</a> - Java, Javascript"
   ],
   [
    "Dark Oberon (<a href=\"http://dark-oberon.sourceforge.net/\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/strategy/dark_oberon.md\">entry</a>)",
@@ -2898,6 +2907,15 @@
    "visual novel, for adults",
    "Ren'py - CC-BY-NC"
   ],
+  [
+   "Stareater (<a href=\"http://stareater4x.blogspot.com/\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/strategy/stareater.md\">entry</a>)",
+   "4X strategy game in space",
+   "<a href=\"https://github.com/subchannel13/Stareater/releases\">Link</a>",
+   "Strategy",
+   "beta / active",
+   "TBS",
+   "<a href=\"https://github.com/subchannel13/Stareater\">Source</a> - C# - GPL-3.0"
+  ],
   [
    "Stars! Nova (<a href=\"https://sourceforge.net/projects/stars-nova/\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/strategy/stars_nova.md\">entry</a>)",
    "Clone of the classic 4X space strategy game Stars!.",
@@ -3609,6 +3627,15 @@
    "gui toolkit",
    "<a href=\"https://github.com/wxWidgets/wxWidgets\">Source</a> - C++, Objective C - Custom"
   ],
+  [
+   "xdigger (<a href=\"https://www.gsp.com/cgi-bin/man.cgi?section=6&topic=xdigger#9\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/action/xdigger.md\">entry</a>)",
+   "(KC85-)Digger game.",
+   "<a href=\"https://launchpad.net/debian/+source/xdigger\">Link</a>",
+   "Action/Arcade",
+   "mature / inactive since 1999",
+   "",
+   "C - GPL-2.0"
+  ],
   [
    "xoreos (<a href=\"https://xoreos.org/\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/framework/xoreos.md\">entry</a>)",
    "Reimplementation of BioWare\u2019s Aurora engine (and..",

games/action/_toc.md

@@ -2,6 +2,7 @@
 
 [comment]: # (start of autogenerated content, do not edit)
 - **[Astromenace](astromenace.md)** (C++, C, GPL-3.0, mature)
+- **[Boulder Dash](boulder_dash.md)** (Java, MIT, mature, inactive since 2015)
 - **[Cat Mother Dead Justice](catmother.md)** (C++, BSD, beta, inactive since 2003)
 - **[CommandoJS](commando_js.md)** (JavaScript, CoffeeScript, MIT, beta, inactive since 2012)
 - **[Epiar](epiar.md)** (C, C++, Lua, GPL-2.0, beta)
@@ -19,5 +20,6 @@
 - **[SuperTuxKart](super_tux_kart.md)** (C++, C, GPL-3.0, mature)
 - **[Tux Racer](tuxracer.md)** (C, GPL-2.0, beta, inactive since 2001)
 - **[Xonotic](xonotic.md)** (C, GPL-3.0, mature)
+- **[xdigger](xdigger.md)** (C, GPL-2.0, mature, inactive since 1999)
 
 [comment]: # (end of autogenerated content)

games/action/boulder_dash.md

@@ -0,0 +1,14 @@
+# Boulder Dash
+
+_Boulder Dash game remake, done in Java._
+
+- Home: https://github.com/valeriansaliou/boulder-dash
+- State: mature, inactive since 2015
+- Download: https://github.com/valeriansaliou/boulder-dash/releases
+- Code repository: https://github.com/valeriansaliou/boulder-dash
+- Code language: Java
+- Code license: MIT
+
+## Building
+
+

games/action/xdigger.md

@@ -0,0 +1,14 @@
+# xdigger
+
+_(KC85-)Digger game._
+
+- Home: https://www.gsp.com/cgi-bin/man.cgi?section=6&topic=xdigger#9
+- State: mature, inactive since 1999
+- Download: https://launchpad.net/debian/+source/xdigger
+- Platform: Linux
+- Code repository: (see download)
+- Code language: C
+- Code license: GPL-2.0
+
+## Building
+

games/statistics.md

@@ -1,17 +1,17 @@
 [comment]: # (autogenerated content, do not edit)
 # Statistics
 
-analyzed 403 entries on 2018-06-29 11:00:44
+analyzed 406 entries on 2018-07-02 13:06:38
 
 ## State
 
-- mature: 238 (59.1%)
+- mature: 240 (59.1%)
-- beta: 165 (40.9%)
+- beta: 166 (40.9%)
-- inactive: 171 (42.4%)
+- inactive: 173 (42.6%)
 
 ##### Inactive State
 
-Zone of Control (2017), 3d.city (2016), Atlantis (2016), Atrinik (2016), Birth of the Empires (2016), DNT (2016), Dark Destiny (2016), Dungeon Monkey Eternal (2016), FreeRCT (2016), Mechanized Assault & eXploration Reloaded  (2016), Polis (2016), WarGame (2016), Dash Engine (2015), Delta Engine (2015), Egoboo (2015), Free heroes2 engine (2015), M.E.W.L. (2015), Project Helena (2015), Roguish (2015), The Epic of Heroes (2015), TwinEngine (2015), Witch Blast (2015), pyORPG (2015), Argentum Online (2014), Battles of Antargis (2014), Deity (2014), Drop Shock (2014), Dune Dynasty (2014), EternalWinterWars (2014), FAR Colony (2014), Hale (2014), Heroes of Wing Commander (2014), Kingdoms (2014), Lips of Suna (2014), Open RPG Maker (2014), Open Tibia (2014), Radakan (2014), SDL Game Engine 2D (2014), Sintel The Game (2014), Summoning Wars (2014), Supremacy (2014), Turious (2014), Xenowar (2014), rpge (2014), sandbox Game Maker (2014), Bos Wars (2013), C-evo (2013), Decker (2013), DemiGod (2013), Dungeon Mapper (2013), Holyspirit (2013), MechCommander 2 Omnitech (2013), Mpango (2013), Murder In The Public Domain (2013), OpenRPG (2013), Phantasy Star Rebirth (2013), Room for Change (2013), Source of Tales (2013), Trinity Reign (2013), VegaTrek (2013), BlackNova Traders (2012), Blitzkrieg (2012), Cataclysm (2012), CommandoJS (2012), Conquests (2012), Dark City (2012), Goblin Camp (2012), Hexwar (2012), PARPG (2012), Solar Empire (2012), Thousand Parsec (2012), UFO2000 (2012), Wizards Magic (2012), Age of Magic (2011), Battlefield Java (2011), Danger from the Deep (2011), Dawn (2011), GalaxyMage Redux (2011), JaNaG - Java Name Generator (2011), KQ Lives (2011), Kobold's Quest 2 (2011), OpenMOO2 (2011), RuneSword II (2011), Skrupel - Tribute Compilation (2011), Song of Albion (2011), Star Control II: The Ur-Quan Masters (2011), UlDunAd (2011), World of Phaos (2011), X-Force: Fight For Destiny (2011), AI Wars (2010), Crimson Fields (2010), Dragon History - Dračí Historie (2010), Endgame: Singularity (2010), Heroes of Wesnoth (2010), Labyrinth of Worlds (2010), Linwarrior 3D (2010), Mercenary Commander (2010), OpenHoMM (2010), Slay (2010), Solaris the Game (2010), SpaceTrader for Java (2010), XSera (2010), 8 Kingdoms (2009), Alien Assault Traders (2009), Annchienta (2009), Ardentryst (2009), Begin2 (2009), Crown and Cutlass (2009), Dark Oberon (2009), Duel Commander (2009), Fall of Imyrin (2009), Fallen Spire (2009), FreeTrain (2009), Glest (2009), Goblin Hack (2009), ORIENT (2009), OpenPop (2009), Secret Mario Chronicles (2009), Space Trader for Windows (2009), World of Heroes (2009), A Planet's Revenge (2008), Attal: Lords of doom (2008), Avanor (2008), Freestars (2008), Mars, Land of No Mercy (2008), Meritous (2008), S.C.O.U.R.G.E. (2008), Scoreos (2008), Sengoku: Warring States of Japan (2008), Space Opera (2008), Armies (2007), Eos, Dawn of Light: A Space Opera (2007), Silvertree (2007), Slash'EM (2007), Underworld Adventures (2007), Arthur's Knights (2006), Betrayer's Moon Tactics (2006), Boson (2006), Eleconics (2006), JQuest (2006), Linley's Dungeon Crawl (2006), Metal Mech (2006), RPDungeon - computer aided role playing (2006), Rogue Clone IV (2006), Visions from the Other Side (2006), GalaxyNG (2005), H-World (2005), LinCity (2005), Spice Trade (2005), WAtomic (2005), Xconq (2005), ZAngband (2005), Colonization too (2004), GM Tools (2004), Gee Whiz (2004), Openglad (2004), Promisance (2004), T-Bots (2004), AntiChess (2003), Cat Mother Dead Justice (2003), Civil (2003), Machinations (2003), Pizza Business (2003), Planetary Hoppers (2003), The Clans (2003), Wargamer (2003), XArchon (2003), Umbra (2002), Greenius' Civil War (2001), Tux Racer (2001), Operation Citadel (2000)
+Zone of Control (2017), 3d.city (2016), Atlantis (2016), Atrinik (2016), Birth of the Empires (2016), DNT (2016), Dark Destiny (2016), Dungeon Monkey Eternal (2016), FreeRCT (2016), Mechanized Assault & eXploration Reloaded  (2016), Polis (2016), WarGame (2016), Boulder Dash (2015), Dash Engine (2015), Delta Engine (2015), Egoboo (2015), Free heroes2 engine (2015), M.E.W.L. (2015), Project Helena (2015), Roguish (2015), The Epic of Heroes (2015), TwinEngine (2015), Witch Blast (2015), pyORPG (2015), Argentum Online (2014), Battles of Antargis (2014), Deity (2014), Drop Shock (2014), Dune Dynasty (2014), EternalWinterWars (2014), FAR Colony (2014), Hale (2014), Heroes of Wing Commander (2014), Kingdoms (2014), Lips of Suna (2014), Open RPG Maker (2014), Open Tibia (2014), Radakan (2014), SDL Game Engine 2D (2014), Sintel The Game (2014), Summoning Wars (2014), Supremacy (2014), Turious (2014), Xenowar (2014), rpge (2014), sandbox Game Maker (2014), Bos Wars (2013), C-evo (2013), Decker (2013), DemiGod (2013), Dungeon Mapper (2013), Holyspirit (2013), MechCommander 2 Omnitech (2013), Mpango (2013), Murder In The Public Domain (2013), OpenRPG (2013), Phantasy Star Rebirth (2013), Room for Change (2013), Source of Tales (2013), Trinity Reign (2013), VegaTrek (2013), BlackNova Traders (2012), Blitzkrieg (2012), Cataclysm (2012), CommandoJS (2012), Conquests (2012), Dark City (2012), Goblin Camp (2012), Hexwar (2012), PARPG (2012), Solar Empire (2012), Thousand Parsec (2012), UFO2000 (2012), Wizards Magic (2012), Age of Magic (2011), Battlefield Java (2011), Danger from the Deep (2011), Dawn (2011), GalaxyMage Redux (2011), JaNaG - Java Name Generator (2011), KQ Lives (2011), Kobold's Quest 2 (2011), OpenMOO2 (2011), RuneSword II (2011), Skrupel - Tribute Compilation (2011), Song of Albion (2011), Star Control II: The Ur-Quan Masters (2011), UlDunAd (2011), World of Phaos (2011), X-Force: Fight For Destiny (2011), AI Wars (2010), Crimson Fields (2010), Dragon History - Dračí Historie (2010), Endgame: Singularity (2010), Heroes of Wesnoth (2010), Labyrinth of Worlds (2010), Linwarrior 3D (2010), Mercenary Commander (2010), OpenHoMM (2010), Slay (2010), Solaris the Game (2010), SpaceTrader for Java (2010), XSera (2010), 8 Kingdoms (2009), Alien Assault Traders (2009), Annchienta (2009), Ardentryst (2009), Begin2 (2009), Crown and Cutlass (2009), Dark Oberon (2009), Duel Commander (2009), Fall of Imyrin (2009), Fallen Spire (2009), FreeTrain (2009), Glest (2009), Goblin Hack (2009), ORIENT (2009), OpenPop (2009), Secret Mario Chronicles (2009), Space Trader for Windows (2009), World of Heroes (2009), A Planet's Revenge (2008), Attal: Lords of doom (2008), Avanor (2008), Freestars (2008), Mars, Land of No Mercy (2008), Meritous (2008), S.C.O.U.R.G.E. (2008), Scoreos (2008), Sengoku: Warring States of Japan (2008), Space Opera (2008), Armies (2007), Eos, Dawn of Light: A Space Opera (2007), Silvertree (2007), Slash'EM (2007), Underworld Adventures (2007), Arthur's Knights (2006), Betrayer's Moon Tactics (2006), Boson (2006), Eleconics (2006), JQuest (2006), Linley's Dungeon Crawl (2006), Metal Mech (2006), RPDungeon - computer aided role playing (2006), Rogue Clone IV (2006), Visions from the Other Side (2006), GalaxyNG (2005), H-World (2005), LinCity (2005), Spice Trade (2005), WAtomic (2005), Xconq (2005), ZAngband (2005), Colonization too (2004), GM Tools (2004), Gee Whiz (2004), Openglad (2004), Promisance (2004), T-Bots (2004), AntiChess (2003), Cat Mother Dead Justice (2003), Civil (2003), Machinations (2003), Pizza Business (2003), Planetary Hoppers (2003), The Clans (2003), Wargamer (2003), XArchon (2003), Umbra (2002), Greenius' Civil War (2001), Tux Racer (2001), Operation Citadel (2000), xdigger (1999)
 
 ## Code Languages
 
@@ -21,14 +21,14 @@ Imperium, Land of Fire, Open AL, Open General, Solaris the Game, VegaTrek, Xconq
 
 ##### Language frequency
 
-- C++ (36.8%)
+- C++ (36.5%)
 - C (19.7%)
-- Java (11.5%)
+- Java (11.6%)
-- Python (8.8%)
+- Python (8.7%)
 - Lua (5.2%)
-- JavaScript (3.8%)
+- JavaScript (3.7%)
 - PHP (3.3%)
-- C# (2.5%)
+- C# (2.7%)
 - Pascal (2.3%)
 - Ren'py (0.8%)
 - Haxe (0.6%)
@@ -44,6 +44,7 @@ Imperium, Land of Fire, Open AL, Open General, Solaris the Game, VegaTrek, Xconq
 - Excel Visual Basic (0.2%)
 - GDScript (0.2%)
 - Groovy (0.2%)
+- Javascript (0.2%)
 - Kotlin (0.2%)
 - Objective C (0.2%)
 - Ruby (0.2%)
@@ -63,9 +64,9 @@ Without license tag: 23 (5.7%)
 
 ##### Licenses frequency
 
-- GPL-2.0 (39.2%)
+- GPL-2.0 (39.1%)
 - GPL-3.0 (25.6%)
-- MIT (11.0%)
+- MIT (11.1%)
 - Custom (5.7%)
 - Apache-2.0 (3.1%)
 - zlib (2.3%)
@@ -98,15 +99,15 @@ Without license tag: 23 (5.7%)
 
 ##### Keywords frequency
 
-- TBS (10.8%)
+- TBS (11.2%)
 - RTS (8.7%)
 - requires original content (7.9%)
 - roguelike (7.9%)
-- remake (7.5%)
+- remake (7.4%)
-- MMO (7.1%)
+- MMO (7.0%)
-- online (7.1%)
+- online (7.0%)
 - tool (5.4%)
-- SP (4.6%)
+- SP (4.5%)
 - MP (3.3%)
 - 3D (2.9%)
 - browser (2.1%)

games/strategy/_toc.md

@@ -37,7 +37,7 @@
 - **[Corsix Theme Hospital](theme_hospital.md)** (Lua, C++, MIT, mature)
 - **[Crimson Fields](crimson_fields.md)** (C++, GPL-2.0, mature, inactive since 2010)
 - **[Crown and Cutlass](crown_and_cutlass.md)** (C++, Custom, beta, inactive since 2009)
-- **[Dark Destiny](dark_destiny.md)** (Java, mature, inactive since 2016)
+- **[Dark Destiny](dark_destiny.md)** (Java, Javascript, mature, inactive since 2016)
 - **[Dark Oberon](dark_oberon.md)** (C++, GPL-2.0, mature, inactive since 2009)
 - **[Devana](devana.md)** (PHP, BSD, beta)
 - **[Drop Shock](drop_shock.md)** (PHP, mature, inactive since 2014)
@@ -132,6 +132,7 @@
 - **[SpaceTrader for Java](space_trader_for_java.md)** (Java, mature, inactive since 2010)
 - **[Spring RTS engine](spring.md)** (C++, C, Lua, GPL-2.0, mature)
 - **[Star Control II: The Ur-Quan Masters](star_control_2.md)** (C, GPL-2.0, mature, inactive since 2011)
+- **[Stareater](stareater.md)** (C#, GPL-3.0, beta)
 - **[Stars! Nova](stars_nova.md)** (C#, GPL-2.0, beta)
 - **[Supremacy](supremacy.md)** (C#, Ms-RL, beta, inactive since 2014)
 - **[T-Bots](t_bots.md)** (C, GPL-2.0, beta, inactive since 2004)

games/strategy/dark_destiny.md

@@ -2,11 +2,13 @@
 
 _Turn-based online space strategy game playable in internet browsers._
 
-- Home: http://www.darkdestiny.at/, http://www.thedarkdestiny.at/portalApp/#/
+- Home: http://www.darkdestiny.at/, http://www.thedarkdestiny.at/portalApp/#/, https://sourceforge.net/projects/darkdestiny/
 - State: mature, inactive since 2016
 - Keywords: MMO
-- Code repository: https://svn.code.sf.net/p/darkdestiny/code/ (svn)
+- Code repository:  https://svn.code.sf.net/p/darkdestiny/code/ (svn)
-- Code language: Java
+- Code language: Java, Javascript
+
+See also https://sourceforge.net/projects/darkdestinytools/
 
 ## Building
 

games/strategy/stareater.md

@@ -0,0 +1,15 @@
+# Stareater
+
+_4X strategy game in space_
+
+- Home: http://stareater4x.blogspot.com/, https://sourceforge.net/projects/zvjezdojedac/
+- State: beta
+- Download: https://github.com/subchannel13/Stareater/releases
+- Platform: Windows
+- Keywords: TBS
+- Code repository: https://github.com/subchannel13/Stareater
+- Code language: C#
+- Code license: GPL-3.0
+
+## Building
+

tools/archives.json

@@ -269,6 +269,7 @@
   "https://github.com/snauts/game-lv.git",
   "https://github.com/solarus-games/solarus.git",
   "https://github.com/spring/spring.git",
+  "https://github.com/subchannel13/Stareater.git",
   "https://github.com/superpowers/superpowers-core.git",
   "https://github.com/supertuxkart/stk-code.git",
   "https://github.com/suprafun/aiwars.git",
@@ -287,6 +288,7 @@
   "https://github.com/unknown-horizons/unknown-horizons.git",
   "https://github.com/unnethack/unnethack.git",
   "https://github.com/urho3d/Urho3D.git",
+  "https://github.com/valeriansaliou/boulder-dash.git",
   "https://github.com/vcmi/vcmi.git",
   "https://github.com/vcosta/derclou.git",
   "https://github.com/vegastrike/Vega-Strike-Engine-Source.git",

tools/lechemindeladam_svn_to_git.py

@@ -0,0 +1,192 @@
+"""
+The svn is too big to be automatically imported to git (and Github) because there are lots of large binary data components.
+Needs a manual solution.
+"""
+
+import json
+
+from utils.utils import *
+
+
+def special_treatment(destination, revision):
+    """
+
+    """
+
+    if revision == 5:
+        shutil.rmtree(os.path.join(destination, 'Holyspirit'))
+
+
+def delete_global_excludes(folder):
+    """
+
+    """
+    for dirpath, dirnames, filenames in os.walk(folder):
+        rel_path = os.path.relpath(dirpath, folder)
+        if rel_path.startswith('.svn'):
+            continue
+        for file in filenames:
+            if file in global_exclude:
+                os.remove(os.path.join(dirpath, file))
+
+
+def delete_empty_directories(folder):
+    """
+
+    """
+    for dirpath, dirnames, filenames in os.walk(folder, topdown=False):
+        rel_path = os.path.relpath(dirpath, folder)
+        if rel_path.startswith('.svn'):
+            continue
+        if not filenames and not dirnames:
+            os.removedirs(dirpath)
+
+
+def list_large_unwanted_files(folder):
+    """
+
+    """
+    output = []
+    for dirpath, dirnames, filenames in os.walk(folder):
+        rel_path = os.path.relpath(dirpath, folder)
+        if rel_path.startswith('.svn'):
+            continue
+        for file in filenames:
+            file_path = os.path.join(dirpath, file)
+            already_listed = False
+            for extension in unwanted_file_extensions:
+                if file.endswith(extension):
+                    output.append(os.path.join(rel_path, file))
+                    already_listed = True
+                    break
+            if not already_listed and os.path.getsize(file_path) > large_file_limit:
+                output.append(os.path.join(rel_path, file))
+    return output
+
+
+def checkout(revision_start, revision_end):
+    """
+
+    """
+    assert revision_end >= revision_start
+
+    for revision in range(revision_start, revision_end + 1):
+        print('checking out revision {}'.format(revision))
+
+        # create destination directory
+        destination = os.path.join(svn_checkout_path, 'r{:04d}'.format(revision))
+        if os.path.exists(destination):
+            shutil.rmtree(destination)
+
+        # checkout
+        start_time = time.time()
+        subprocess_run(['svn', 'checkout', '-r{}'.format(revision), svn_url, destination])
+        print('checkout took {}s'.format(time.time() - start_time))
+
+        # sanitation (delete files from global exclude list)
+        delete_global_excludes(destination)
+
+        # list unwanted files
+        unwanted_files = list_large_unwanted_files(destination)
+        if unwanted_files:
+            text = json.dumps(unwanted_files, indent=1)
+            write_text(os.path.join(svn_checkout_path, 'r{:04d}_unwanted_files.json'.format(revision)), text)
+
+        # delete empty directories
+        delete_empty_directories(destination)
+
+        # special treatment
+        special_treatment(destination, revision)
+
+
+def initialize_git():
+    """
+
+    """
+    # git init
+    os.mkdir(git_path)
+    os.chdir(git_path)
+    subprocess_run(['git', 'init'])
+    subprocess_run(['git', 'config', 'user.name', 'Trilarion'])
+    subprocess_run(['git', 'config', 'user.email', 'Trilarion@users.noreply.gitlab.com'])
+
+
+def combine_log_messages(msg):
+    """
+
+    """
+    # throw out all empty ones
+    msg = [x.strip() for x in msg if x]
+    # combine again
+    msg = "\r\n".join(msg)
+    return msg
+
+
+def read_logs():
+    """
+    Probably regular expressions would have worked too.
+    """
+    # read log
+    print('read all log messages')
+    os.chdir(svn_checkout_path)
+    start_time = time.time()
+    log = subprocess_run(['svn', 'log', svn_url], display=False)
+    print('read log took {}s'.format(time.time() - start_time))
+    # process log
+    log = log.split('\r\n------------------------------------------------------------------------\r\n')
+    # not the last one
+    log = log[:-2]
+    print('{} log entries'.format(len(log)))
+
+    # process log entries
+    log = [x.split('\r\n') for x in log]
+
+    # the first one still contains an additional "---" elements
+    log[0] = log[0][1:]
+
+    # split the first line
+    info = [x[0].split('|') for x in log]
+
+    # get the revision
+    revision = [int(x[0][1:]) for x in info]
+
+    author = [x[1].strip() for x in info]
+    unique_authors = list(set(author))
+    unique_authors.sort()
+
+    date = [x[2].strip() for x in info]
+    msg = [combine_log_messages(x[2:]) for x in log]
+    logs = list(zip(revision, author, date, msg))
+    logs.sort(key=lambda x: x[0])
+    return logs, unique_authors
+
+
+if __name__ == "__main__":
+
+    global_exclude = ['Thumbs.db']
+    unwanted_file_extensions = ['.exe', '.dll']
+    large_file_limit = 1e6  # in bytes
+
+    # base path is the directory containing this file
+    base_path = os.path.abspath(os.path.dirname(__file__))
+    print('base path={}'.format(base_path))
+
+    # derived paths
+    svn_checkout_path = os.path.join(base_path, 'svn_checkout')
+    if not os.path.exists(svn_checkout_path):
+        os.mkdir(svn_checkout_path)
+    git_path = os.path.join(base_path, 'lechemindeladam')
+    # if not os.path.exists(git_path):
+    #    initialize_git()
+
+    # svn url
+    svn_url = "https://svn.code.sf.net/p/lechemindeladam/code/"
+
+    # read logs
+    # logs, authors = read_logs()
+    # text = json.dumps(logs, indent=1)
+    # write_text(os.path.join(svn_checkout_path, 'logs.json'), text)
+    # text = json.dumps(authors, indent=1)
+    # write_text(os.path.join(svn_checkout_path, 'authors.json'), text)
+
+    checkout(1, 50)

tools/utils/utils.py

@@ -146,7 +146,7 @@ def determine_latest_last_modified_date(folder):
     return latest_last_modified
 
 
-def subprocess_run(cmd):
+def subprocess_run(cmd, display=True):
     """
     Runs a cmd via subprocess and displays the std output in case of success or the std error output in case of failure
     where it also stops execution.
@@ -154,10 +154,11 @@ def subprocess_run(cmd):
     result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
     if result.returncode:
         print("error {} in call {}".format(result.returncode, cmd))
-        print(result.stderr.decode('ascii'))
+        print(result.stderr.decode('cp1252'))
         sys.exit(-1)
-    else:
+    if display:
-        print('  output: {}'.format(result.stdout.decode('ascii')))
+        print('  output: {}'.format(result.stdout.decode('cp1252')))
+    return result.stdout.decode('cp1252')
 
 
 def copy_tree(source, destination):