added and removed entries, conversion of holy spirit

README.md

@@ -8,9 +8,9 @@
 - **[Adventure & Visual novels](games/adventure/_toc.md)** (15)
 - **[Engines & Frameworks](games/framework/_toc.md)** (52)
 - **[Libraries](games/library/_toc.md)** (22)
-- **[Role Playing](games/rpg/_toc.md)** (105)
+- **[Role Playing](games/rpg/_toc.md)** (106)
 - **[Simulation](games/simulation/_toc.md)** (15)
-- **[Strategy](games/strategy/_toc.md)** (165)
+- **[Strategy](games/strategy/_toc.md)** (164)
 - **[Tools & Manuals](games/misc/_toc.md)** (12)
 
 [comment]: # (end of autogenerated content)

docs/data.json

@@ -297,6 +297,15 @@
    "",
    "<a href=\"https://git.code.sf.net/p/battlefieldjava/git\">Source</a> - Java"
   ],
+  [
+   "Battleround (<a href=\"https://github.com/Leejjon/Battleround\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/rpg/battleround.md\">entry</a>)",
+   "Round based RPG.",
+   "",
+   "Role Playing",
+   "beta / inactive since 2015",
+   "",
+   "<a href=\"https://github.com/Leejjon/Battleround\">Source</a> - Java - Apache-2.0"
+  ],
   [
    "Battles of Antargis (<a href=\"https://plus.google.com/101003433246259562872\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/strategy/battles_of_antargis.md\">entry</a>)",
    "A real-time-strategy (RTS) game, which is a mixture of 3..",
@@ -2428,7 +2437,7 @@
    "Strategy",
    "mature / inactive since 2004",
    "",
-   "<a href=\"https://gitlab.com/Trilarion/promisance.git\">Source</a> - PHP - GPL-2.0"
+   "<a href=\"https://gitlab.com/Trilarion/promisance-cvs.git\">Source</a> - PHP - GPL-2.0"
   ],
   [
    "Pthreads-win32 (<a href=\"https://sourceware.org/pthreads-win32/\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/library/pthreads_win32.md\">entry</a>)",
@@ -3267,15 +3276,6 @@
    "",
    "<a href=\"https://github.com/freeors/War-Of-Kingdom\">Source</a> - C, C++ - GPL-2.0"
   ],
-  [
-   "WarGame (<a href=\"https://archive.codeplex.com/?p=wargame\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/strategy/wargame.md\">entry</a>)",
-   "Turn based strategy game for Windows, Mac OS X, Linux, and..",
-   "",
-   "Strategy",
-   "beta / inactive since 2016",
-   "",
-   "Unity, C# - CDDL"
-  ],
   [
    "Wargamer (<a href=\"https://sourceforge.net/projects/wargamer/\">home</a>, <a href=\"https://github.com/Trilarion/opensourcegames/blob/master/games/strategy/wargamer.md\">entry</a>)",
    "Open source development of the game Wargamer:Napoleon 1813..",

games/rpg/_toc.md

@@ -8,6 +8,7 @@
 - **[Arx Libertatis](arx_libertatis.md)** (C++, GPL-3.0, mature)
 - **[Atrinik](atrinik.md)** (C, Python, GPL-2.0, mature, inactive since 2016)
 - **[Avanor](avanor.md)** (C++, Lua, GPL-2.0, beta, inactive since 2008)
+- **[Battleround](battleround.md)** (Java, Apache-2.0, beta, inactive since 2015)
 - **[Betrayer's Moon Tactics](betrayer_moon_tactics.md)** (JavaScript, GPL-2.0, beta, inactive since 2006)
 - **[BrowserQuest](browserquest.md)** (JavaScript, MPL-2.0, mature)
 - **[Castle of the Winds](castle_of_the_winds.md)** (JavaScript, MIT, beta)

games/rpg/battleround.md

@@ -0,0 +1,17 @@
+# Battleround
+
+_Round based RPG._
+
+- Home: https://github.com/Leejjon/Battleround
+- State: beta, inactive since 2015
+- Platform: Android
+- Code repository: https://github.com/Leejjon/Battleround
+- Code language: Java
+- Code license: Apache-2.0
+- Code dependencies: Libgdx
+
+May be more like alpha.
+
+## Building
+
+- Build system: Maven

games/statistics.md

@@ -1,7 +1,7 @@
 [comment]: # (autogenerated content, do not edit)
 # Statistics
 
-analyzed 406 entries on 2018-07-02 13:06:38
+analyzed 406 entries on 2018-07-03 11:44:25
 
 ## State
 
@@ -11,7 +11,7 @@ analyzed 406 entries on 2018-07-02 13:06:38
 
 ##### Inactive State
 
-Zone of Control (2017), 3d.city (2016), Atlantis (2016), Atrinik (2016), Birth of the Empires (2016), DNT (2016), Dark Destiny (2016), Dungeon Monkey Eternal (2016), FreeRCT (2016), Mechanized Assault & eXploration Reloaded  (2016), Polis (2016), WarGame (2016), Boulder Dash (2015), Dash Engine (2015), Delta Engine (2015), Egoboo (2015), Free heroes2 engine (2015), M.E.W.L. (2015), Project Helena (2015), Roguish (2015), The Epic of Heroes (2015), TwinEngine (2015), Witch Blast (2015), pyORPG (2015), Argentum Online (2014), Battles of Antargis (2014), Deity (2014), Drop Shock (2014), Dune Dynasty (2014), EternalWinterWars (2014), FAR Colony (2014), Hale (2014), Heroes of Wing Commander (2014), Kingdoms (2014), Lips of Suna (2014), Open RPG Maker (2014), Open Tibia (2014), Radakan (2014), SDL Game Engine 2D (2014), Sintel The Game (2014), Summoning Wars (2014), Supremacy (2014), Turious (2014), Xenowar (2014), rpge (2014), sandbox Game Maker (2014), Bos Wars (2013), C-evo (2013), Decker (2013), DemiGod (2013), Dungeon Mapper (2013), Holyspirit (2013), MechCommander 2 Omnitech (2013), Mpango (2013), Murder In The Public Domain (2013), OpenRPG (2013), Phantasy Star Rebirth (2013), Room for Change (2013), Source of Tales (2013), Trinity Reign (2013), VegaTrek (2013), BlackNova Traders (2012), Blitzkrieg (2012), Cataclysm (2012), CommandoJS (2012), Conquests (2012), Dark City (2012), Goblin Camp (2012), Hexwar (2012), PARPG (2012), Solar Empire (2012), Thousand Parsec (2012), UFO2000 (2012), Wizards Magic (2012), Age of Magic (2011), Battlefield Java (2011), Danger from the Deep (2011), Dawn (2011), GalaxyMage Redux (2011), JaNaG - Java Name Generator (2011), KQ Lives (2011), Kobold's Quest 2 (2011), OpenMOO2 (2011), RuneSword II (2011), Skrupel - Tribute Compilation (2011), Song of Albion (2011), Star Control II: The Ur-Quan Masters (2011), UlDunAd (2011), World of Phaos (2011), X-Force: Fight For Destiny (2011), AI Wars (2010), Crimson Fields (2010), Dragon History - Dračí Historie (2010), Endgame: Singularity (2010), Heroes of Wesnoth (2010), Labyrinth of Worlds (2010), Linwarrior 3D (2010), Mercenary Commander (2010), OpenHoMM (2010), Slay (2010), Solaris the Game (2010), SpaceTrader for Java (2010), XSera (2010), 8 Kingdoms (2009), Alien Assault Traders (2009), Annchienta (2009), Ardentryst (2009), Begin2 (2009), Crown and Cutlass (2009), Dark Oberon (2009), Duel Commander (2009), Fall of Imyrin (2009), Fallen Spire (2009), FreeTrain (2009), Glest (2009), Goblin Hack (2009), ORIENT (2009), OpenPop (2009), Secret Mario Chronicles (2009), Space Trader for Windows (2009), World of Heroes (2009), A Planet's Revenge (2008), Attal: Lords of doom (2008), Avanor (2008), Freestars (2008), Mars, Land of No Mercy (2008), Meritous (2008), S.C.O.U.R.G.E. (2008), Scoreos (2008), Sengoku: Warring States of Japan (2008), Space Opera (2008), Armies (2007), Eos, Dawn of Light: A Space Opera (2007), Silvertree (2007), Slash'EM (2007), Underworld Adventures (2007), Arthur's Knights (2006), Betrayer's Moon Tactics (2006), Boson (2006), Eleconics (2006), JQuest (2006), Linley's Dungeon Crawl (2006), Metal Mech (2006), RPDungeon - computer aided role playing (2006), Rogue Clone IV (2006), Visions from the Other Side (2006), GalaxyNG (2005), H-World (2005), LinCity (2005), Spice Trade (2005), WAtomic (2005), Xconq (2005), ZAngband (2005), Colonization too (2004), GM Tools (2004), Gee Whiz (2004), Openglad (2004), Promisance (2004), T-Bots (2004), AntiChess (2003), Cat Mother Dead Justice (2003), Civil (2003), Machinations (2003), Pizza Business (2003), Planetary Hoppers (2003), The Clans (2003), Wargamer (2003), XArchon (2003), Umbra (2002), Greenius' Civil War (2001), Tux Racer (2001), Operation Citadel (2000), xdigger (1999)
+Zone of Control (2017), 3d.city (2016), Atlantis (2016), Atrinik (2016), Birth of the Empires (2016), DNT (2016), Dark Destiny (2016), Dungeon Monkey Eternal (2016), FreeRCT (2016), Mechanized Assault & eXploration Reloaded  (2016), Polis (2016), Battleround (2015), Boulder Dash (2015), Dash Engine (2015), Delta Engine (2015), Egoboo (2015), Free heroes2 engine (2015), M.E.W.L. (2015), Project Helena (2015), Roguish (2015), The Epic of Heroes (2015), TwinEngine (2015), Witch Blast (2015), pyORPG (2015), Argentum Online (2014), Battles of Antargis (2014), Deity (2014), Drop Shock (2014), Dune Dynasty (2014), EternalWinterWars (2014), FAR Colony (2014), Hale (2014), Heroes of Wing Commander (2014), Kingdoms (2014), Lips of Suna (2014), Open RPG Maker (2014), Open Tibia (2014), Radakan (2014), SDL Game Engine 2D (2014), Sintel The Game (2014), Summoning Wars (2014), Supremacy (2014), Turious (2014), Xenowar (2014), rpge (2014), sandbox Game Maker (2014), Bos Wars (2013), C-evo (2013), Decker (2013), DemiGod (2013), Dungeon Mapper (2013), Holyspirit (2013), MechCommander 2 Omnitech (2013), Mpango (2013), Murder In The Public Domain (2013), OpenRPG (2013), Phantasy Star Rebirth (2013), Room for Change (2013), Source of Tales (2013), Trinity Reign (2013), VegaTrek (2013), BlackNova Traders (2012), Blitzkrieg (2012), Cataclysm (2012), CommandoJS (2012), Conquests (2012), Dark City (2012), Goblin Camp (2012), Hexwar (2012), PARPG (2012), Solar Empire (2012), Thousand Parsec (2012), UFO2000 (2012), Wizards Magic (2012), Age of Magic (2011), Battlefield Java (2011), Danger from the Deep (2011), Dawn (2011), GalaxyMage Redux (2011), JaNaG - Java Name Generator (2011), KQ Lives (2011), Kobold's Quest 2 (2011), OpenMOO2 (2011), RuneSword II (2011), Skrupel - Tribute Compilation (2011), Song of Albion (2011), Star Control II: The Ur-Quan Masters (2011), UlDunAd (2011), World of Phaos (2011), X-Force: Fight For Destiny (2011), AI Wars (2010), Crimson Fields (2010), Dragon History - Dračí Historie (2010), Endgame: Singularity (2010), Heroes of Wesnoth (2010), Labyrinth of Worlds (2010), Linwarrior 3D (2010), Mercenary Commander (2010), OpenHoMM (2010), Slay (2010), Solaris the Game (2010), SpaceTrader for Java (2010), XSera (2010), 8 Kingdoms (2009), Alien Assault Traders (2009), Annchienta (2009), Ardentryst (2009), Begin2 (2009), Crown and Cutlass (2009), Dark Oberon (2009), Duel Commander (2009), Fall of Imyrin (2009), Fallen Spire (2009), FreeTrain (2009), Glest (2009), Goblin Hack (2009), ORIENT (2009), OpenPop (2009), Secret Mario Chronicles (2009), Space Trader for Windows (2009), World of Heroes (2009), A Planet's Revenge (2008), Attal: Lords of doom (2008), Avanor (2008), Freestars (2008), Mars, Land of No Mercy (2008), Meritous (2008), S.C.O.U.R.G.E. (2008), Scoreos (2008), Sengoku: Warring States of Japan (2008), Space Opera (2008), Armies (2007), Eos, Dawn of Light: A Space Opera (2007), Silvertree (2007), Slash'EM (2007), Underworld Adventures (2007), Arthur's Knights (2006), Betrayer's Moon Tactics (2006), Boson (2006), Eleconics (2006), JQuest (2006), Linley's Dungeon Crawl (2006), Metal Mech (2006), RPDungeon - computer aided role playing (2006), Rogue Clone IV (2006), Visions from the Other Side (2006), GalaxyNG (2005), H-World (2005), LinCity (2005), Spice Trade (2005), WAtomic (2005), Xconq (2005), ZAngband (2005), Colonization too (2004), GM Tools (2004), Gee Whiz (2004), Openglad (2004), Promisance (2004), T-Bots (2004), AntiChess (2003), Cat Mother Dead Justice (2003), Civil (2003), Machinations (2003), Pizza Business (2003), Planetary Hoppers (2003), The Clans (2003), Wargamer (2003), XArchon (2003), Umbra (2002), Greenius' Civil War (2001), Tux Racer (2001), Operation Citadel (2000), xdigger (1999)
 
 ## Code Languages
 
@@ -21,14 +21,14 @@ Imperium, Land of Fire, Open AL, Open General, Solaris the Game, VegaTrek, Xconq
 
 ##### Language frequency
 
-- C++ (36.5%)
+- C++ (36.6%)
-- C (19.7%)
+- C (19.8%)
-- Java (11.6%)
+- Java (11.9%)
 - Python (8.7%)
 - Lua (5.2%)
 - JavaScript (3.7%)
 - PHP (3.3%)
-- C# (2.7%)
+- C# (2.5%)
 - Pascal (2.3%)
 - Ren'py (0.8%)
 - Haxe (0.6%)
@@ -52,7 +52,6 @@ Imperium, Land of Fire, Open AL, Open General, Solaris the Game, VegaTrek, Xconq
 - Script (0.2%)
 - Text (0.2%)
 - Typescript (0.2%)
-- Unity (0.2%)
 - VB (0.2%)
 - Visual Basic (0.2%)
 
@@ -68,7 +67,7 @@ Without license tag: 23 (5.7%)
 - GPL-3.0 (25.6%)
 - MIT (11.1%)
 - Custom (5.7%)
-- Apache-2.0 (3.1%)
+- Apache-2.0 (3.4%)
 - zlib (2.3%)
 - BSD (2.1%)
 - GPL (1.3%)
@@ -85,7 +84,6 @@ Without license tag: 23 (5.7%)
 - CC-BY-NC-SA (0.3%)
 - CC-BY-SA-3 (0.3%)
 - CC0 (0.3%)
-- CDDL (0.3%)
 - Educational (0.3%)
 - Freeware (0.3%)
 - ISC (0.3%)

games/strategy/_toc.md

@@ -149,7 +149,6 @@
 - **[VCMI Project](vcmi.md)** (C++, GPL-2.0, mature)
 - **[WAtomic](watomic.md)** (Pascal, GPL-2.0, mature, inactive since 2005)
 - **[War Of Kingdom](war_of_kingdom.md)** (C, C++, GPL-2.0, beta)
-- **[WarGame](wargame.md)** (Unity, C#, CDDL, beta, inactive since 2016)
 - **[Wargamer](wargamer.md)** (C++, GPL-2.0, mature, inactive since 2003)
 - **[Wargus](wargus.md)** (C++, Lua, GPL-2.0, mature)
 - **[Warzone 2100](warzone_2100.md)** (C++, GPL-2.0, mature)

games/strategy/c_evo.md

@@ -10,4 +10,6 @@ _C-evo is a freeware empire building game for Windows._
 - Code repository: (see download http://www.c-evo.org/files/download.php?cevosrc.cevosrc.zip)
 - Code language: Pascal
 
+See also https://github.com/vn971/cevo
+
 ## Building

games/strategy/promisance.md

@@ -5,7 +5,7 @@ _Web-based turn-based strategy game._
 - Home: https://sourceforge.net/projects/promisance/, http://promisance.sourceforge.net/
 - Download: https://sourceforge.net/projects/promisance/files
 - State: mature, inactive since 2004
-- Code repository: https://gitlab.com/Trilarion/promisance.git (backup of cvs), http://promisance.cvs.sourceforge.net/ (cvs)
+- Code repository: https://gitlab.com/Trilarion/promisance-cvs.git (backup of cvs), http://promisance.cvs.sourceforge.net/ (cvs)
 - Code language: PHP
 - Code license: GPL-2.0
 

games/strategy/wargame.md

@@ -1,14 +0,0 @@
-# WarGame
-
-_Turn based strategy game for Windows, Mac OS X, Linux, and Android._
-
-- Home: https://archive.codeplex.com/?p=wargame
-- State: beta, inactive since 2016
-- Code repository: (see download)
-- Code language: Unity, C#
-- Code license: CDDL
-
-CDDL = Common Development License
-
-## Building
-

tools/archives.json

@@ -77,6 +77,7 @@
   "https://github.com/Illarion-eV/Illarion-Server.git",
   "https://github.com/Kromster80/kam_remake.git",
   "https://github.com/LWJGL/lwjgl3.git",
+  "https://github.com/Leejjon/Battleround.git",
   "https://github.com/LibreGamesArchive/galaxymageredux.git",
   "https://github.com/LibreGamesArchive/silvertree.git",
   "https://github.com/LionsPhil/mewl.git",
@@ -336,7 +337,7 @@
   "https://gitlab.com/Trilarion/phaosrpg.git",
   "https://gitlab.com/Trilarion/pizza-business.git",
   "https://gitlab.com/Trilarion/planets.git",
-  "https://gitlab.com/Trilarion/promisance.git",
+  "https://gitlab.com/Trilarion/promisance-cvs.git",
   "https://gitlab.com/Trilarion/pymapper.git",
   "https://gitlab.com/Trilarion/rogueclone.git",
   "https://gitlab.com/Trilarion/sentient-storage.git",

tools/lechemindeladam_svn_to_git.py

@@ -1,6 +1,8 @@
 """
 The svn is too big to be automatically imported to git (and Github) because there are lots of large binary data components.
 Needs a manual solution.
+
+TODO use git lfs migrate later on the elements
 """
 
 import json
@@ -13,8 +15,46 @@ def special_treatment(destination, revision):
 
     """
 
-    if revision == 5:
+    # copy all important files from Holyspirit/Holyspirit and delete it
-        shutil.rmtree(os.path.join(destination, 'Holyspirit'))
+    if 5 <= revision <= 330:
+        source = os.path.join(destination, 'Holyspirit', 'Holyspirit')
+        if os.path.isdir(source):
+            if revision >= 8:
+                shutil.copytree(os.path.join(source, 'Data'), os.path.join(destination, 'Data'))
+            files = [x for x in os.listdir(source) if x.endswith('.txt')]
+            for file in files:
+                shutil.copy(os.path.join(source, file), destination)
+            # remove it
+            shutil.rmtree(os.path.join(destination, 'Holyspirit'))
+
+    if 337 <= revision <= 400:
+        source = os.path.join(destination, 'Holyspirit')
+        if os.path.isdir(source):
+            data = os.path.join(source, 'Data')
+            if os.path.isdir(data):
+                shutil.copytree(data, os.path.join(destination, 'Data'))
+                files = [x for x in os.listdir(source) if x.endswith('.txt') or x.endswith('.conf')]
+                for file in files:
+                    shutil.copy(os.path.join(source, file), destination)
+            # remove it
+            shutil.rmtree(source)
+
+    # delete all *.dll, *.exe in base folder
+    if 3 <= revision <= 9:
+        files = os.listdir(destination)
+        for file in files:
+            if file.endswith('.exe') or file.endswith('.dll'):
+                os.remove(os.path.join(destination, file))
+
+    if 42 <= revision <= 43:
+        folder = os.path.join(destination, 'Cross')
+        if os.path.isdir(folder):
+            shutil.rmtree(folder)
+
+    if 374 <= revision:
+        folder = os.path.join(destination, 'Photos')
+        if os.path.isdir(folder):
+            shutil.rmtree(folder)
 
 
 def delete_global_excludes(folder):
@@ -23,8 +63,6 @@ def delete_global_excludes(folder):
     """
     for dirpath, dirnames, filenames in os.walk(folder):
         rel_path = os.path.relpath(dirpath, folder)
-        if rel_path.startswith('.svn'):
-            continue
         for file in filenames:
             if file in global_exclude:
                 os.remove(os.path.join(dirpath, file))
@@ -36,8 +74,6 @@ def delete_empty_directories(folder):
     """
     for dirpath, dirnames, filenames in os.walk(folder, topdown=False):
         rel_path = os.path.relpath(dirpath, folder)
-        if rel_path.startswith('.svn'):
-            continue
         if not filenames and not dirnames:
             os.removedirs(dirpath)
 
@@ -49,18 +85,16 @@ def list_large_unwanted_files(folder):
     output = []
     for dirpath, dirnames, filenames in os.walk(folder):
         rel_path = os.path.relpath(dirpath, folder)
-        if rel_path.startswith('.svn'):
-            continue
         for file in filenames:
             file_path = os.path.join(dirpath, file)
             already_listed = False
             for extension in unwanted_file_extensions:
                 if file.endswith(extension):
-                    output.append(os.path.join(rel_path, file))
+                    output.append(os.path.join(rel_path, file) + ' ' + str(os.path.getsize(file_path)))
                     already_listed = True
                     break
             if not already_listed and os.path.getsize(file_path) > large_file_limit:
-                output.append(os.path.join(rel_path, file))
+                output.append(os.path.join(rel_path, file) + ' ' + str(os.path.getsize(file_path)))
     return output
 
 
@@ -80,23 +114,51 @@ def checkout(revision_start, revision_end):
 
         # checkout
         start_time = time.time()
-        subprocess_run(['svn', 'checkout', '-r{}'.format(revision), svn_url, destination])
+        subprocess_run(['svn', 'export', '-r{}'.format(revision), svn_url, destination])
-        print('checkout took {}s'.format(time.time() - start_time))
+        print('checkout took {:.1f}s'.format(time.time() - start_time))
 
-        # sanitation (delete files from global exclude list)
+        # copy to backup
+        # backup_path = os.path.join(svn_backup_path, 'r{:04d}'.format(revision))
+        # if not os.path.exists(backup_path):
+        # shutil.copytree(destination, backup_path)
+
+
+def fix_revision(revision_start, revision_end):
+    """
+
+    """
+    assert revision_end >= revision_start
+
+    unwanted_files = {}
+    sizes = {}
+
+    for revision in range(revision_start, revision_end + 1):
+        print('fixing revision {}'.format(revision))
+
+        # destination directory
+        destination = os.path.join(svn_checkout_path, 'r{:04d}'.format(revision))
+        if not os.path.exists(destination):
+            raise RuntimeError('cannot fix revision {}, directory does not exist'.format(revision))
+
+        # special treatment
+        special_treatment(destination, revision)
+
+        # delete files from global exclude list
         delete_global_excludes(destination)
 
         # list unwanted files
-        unwanted_files = list_large_unwanted_files(destination)
+        unwanted_files[revision] = list_large_unwanted_files(destination)
-        if unwanted_files:
-            text = json.dumps(unwanted_files, indent=1)
-            write_text(os.path.join(svn_checkout_path, 'r{:04d}_unwanted_files.json'.format(revision)), text)
 
         # delete empty directories
         delete_empty_directories(destination)
 
-        # special treatment
+        # size of resulting folder
-        special_treatment(destination, revision)
+        sizes[revision] = folder_size(destination)
+
+    text = json.dumps(unwanted_files, indent=1)
+    write_text(os.path.join(svn_checkout_path, 'unwanted_files.json'.format(revision)), text)
+    text = json.dumps(sizes, indent=1)
+    write_text(os.path.join(svn_checkout_path, 'folder_sizes.json'.format(revision)), text)
 
 
 def initialize_git():
@@ -131,7 +193,7 @@ def read_logs():
     os.chdir(svn_checkout_path)
     start_time = time.time()
     log = subprocess_run(['svn', 'log', svn_url], display=False)
-    print('read log took {}s'.format(time.time() - start_time))
+    print('read log took {:.1f}s'.format(time.time() - start_time))
     # process log
     log = log.split('\r\n------------------------------------------------------------------------\r\n')
     # not the last one
@@ -161,6 +223,60 @@ def read_logs():
     return logs, unique_authors
 
 
+def gitify(revision_start, revision_end):
+    """
+
+    """
+    assert revision_end >= revision_start
+
+    for revision in range(revision_start, revision_end + 1):
+        print('adding revision {} to git'.format(revision))
+
+        # svn folder
+        svn_folder = os.path.join(svn_checkout_path, 'r{:04d}'.format(revision))
+        if not os.path.exists(svn_folder):
+            raise RuntimeError('cannot add revision {}, directory does not exist'.format(revision))
+
+        # clear git path
+        print('git clear path')
+        while True:
+            try:
+                git_clear_path(git_path)
+                break
+            except PermissionError as e:
+                print(e)
+                # wait a bit
+                time.sleep(1)
+
+        # copy source files to git path
+        print('copy to git')
+        copy_tree(svn_folder, git_path)
+
+        os.chdir(git_path)
+
+        # update the git index (add unstaged, remove deleted, ...)
+        print('git add')
+        subprocess_run(['git', 'add', '--all'])
+
+        # check if there is something to commit
+        status = subprocess_run(['git', 'status', '--porcelain'])
+        if not status:
+            print(' nothing to commit for revision {}, will skip'.format(revision))
+            continue
+
+        # perform the commit
+        print('git commit')
+        log = logs[revision]  # revision, author, date, message
+        message = log[3] + '\r\nsvn-revision: {}'.format(revision)
+        print('  message "{}"'.format(message))
+        author = authors[log[1]]
+        author = '{} <{}>'.format(*author)
+        cmd = ['git', 'commit', '--allow-empty-message', '--message={}'.format(message), '--author={}'.format(author),
+               '--date={}'.format(log[2])]
+        print('  cmd: {}'.format(' '.join(cmd)))
+        subprocess_run(cmd)
+
+
 if __name__ == "__main__":
 
     global_exclude = ['Thumbs.db']
@@ -168,16 +284,19 @@ if __name__ == "__main__":
     large_file_limit = 1e6  # in bytes
 
     # base path is the directory containing this file
-    base_path = os.path.abspath(os.path.dirname(__file__))
+    base_path = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'conversion')
     print('base path={}'.format(base_path))
 
     # derived paths
-    svn_checkout_path = os.path.join(base_path, 'svn_checkout')
+    svn_checkout_path = os.path.join(base_path, 'svn')
     if not os.path.exists(svn_checkout_path):
         os.mkdir(svn_checkout_path)
+    svn_backup_path = os.path.join(base_path, 'svn_backup')
+    if not os.path.exists(svn_backup_path):
+        os.mkdir(svn_backup_path)
     git_path = os.path.join(base_path, 'lechemindeladam')
-    # if not os.path.exists(git_path):
+    if not os.path.exists(git_path):
-    #    initialize_git()
+        initialize_git()
 
     # svn url
     svn_url = "https://svn.code.sf.net/p/lechemindeladam/code/"
@@ -185,8 +304,31 @@ if __name__ == "__main__":
     # read logs
     # logs, authors = read_logs()
     # text = json.dumps(logs, indent=1)
-    # write_text(os.path.join(svn_checkout_path, 'logs.json'), text)
+    # write_text(os.path.join(base_path, 'logs.json'), text)
     # text = json.dumps(authors, indent=1)
-    # write_text(os.path.join(svn_checkout_path, 'authors.json'), text)
+    # write_text(os.path.join(base_path, 'authors.json'), text)
+    text = read_text(os.path.join(base_path, 'logs.json'))
+    logs = json.loads(text)
+    logs = {x[0]: x for x in logs}  # dictionary
+    text = read_text(os.path.join(base_path, 'authors.json'))
+    authors = json.loads(text)  # should be a dictionary: svn-author: [git-author, git-email]
 
-    checkout(1, 50)
+    # the steps
+    # checkout(1, 50)
+    # fix_revision(1, 50)
+    # gitify(4, 50)
+
+    # checkout(51, 100)
+    # checkout(101, 200)
+
+    # fix_revision(51, 200)
+
+    # gitify(51, 200)
+
+    # checkout(201, 400)
+    # fix_revision(201, 400)
+    # gitify(201, 400)
+
+    # checkout(401, 800)
+
+    checkout(493, 800)

tools/utils/utils.py

@@ -2,14 +2,14 @@
 Utilities for the tools.
 """
 
-import sys
 import os
-import time
-import zipfile
-import tarfile
-import subprocess
 import shutil
+import subprocess
+import sys
+import tarfile
+import time
 import urllib.request
+import zipfile
 
 
 def read_text(file):
@@ -154,6 +154,7 @@ def subprocess_run(cmd, display=True):
     result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
     if result.returncode:
         print("error {} in call {}".format(result.returncode, cmd))
+        print(result.stdout.decode('cp1252'))
         print(result.stderr.decode('cp1252'))
         sys.exit(-1)
     if display:
@@ -170,7 +171,8 @@ def copy_tree(source, destination):
     # distutils.dir_util.copy_tree(archive_path, git_path)
     for dirpath, dirnames, filenames in os.walk(source):
         # first create all the directory on destination
-        directories_to_be_created = [os.path.join(destination, os.path.relpath(os.path.join(dirpath, x), source)) for x in dirnames]
+        directories_to_be_created = [os.path.join(destination, os.path.relpath(os.path.join(dirpath, x), source)) for x
+                                     in dirnames]
         for directory in directories_to_be_created:
             os.makedirs(directory, exist_ok=True)
         # second copy all the files
@@ -190,3 +192,18 @@ def download_url(url, destination):
     with urllib.request.urlopen(url) as response:
         with open(destination, 'wb') as f:
             shutil.copyfileobj(response, f)
+
+
+def git_clear_path(git_path):
+    """
+    Clears all in a path except the '.git' directory
+    """
+    for item in os.listdir(git_path):
+        # ignore '.git
+        if item == '.git':
+            continue
+        item = os.path.join(git_path, item)
+        if os.path.isdir(item):
+            shutil.rmtree(item)
+        else:
+            os.remove(item)