AnotherFoxGuy/polyvox
1
0
Fork 0
Code Issues Pull Requests Activity

Ran documentation through spell checker.

Browse Source
This commit is contained in:
David Williams 2012-11-18 14:23:33 +01:00
parent 64cb4f654f
commit 3b1a3f9536
6 changed files with 43 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
CHANGELOG.txt
View File

@ -1,14 +1,14 @@
Changes for PolyVox version 0.2
===============================
This is the first revision for which we have produced a changelog, as we are now trying to streamline our development and release process. Hence this changelog entry is not going to be as structured as we hope they will be in the future. We're writing it from memory, rather than having carefully kept track of all the relevent changes as we made them.
This is the first revision for which we have produced a changelog, as we are now trying to streamline our development and release process. Hence this changelog entry is not going to be as structured as we hope they will be in the future. We're writing it from memory, rather than having carefully kept track of all the relevant changes as we made them.
Deprecated functionality
------------------------
The following functionality is considered deprecated in this release of PolyVox and will probably be removed in future versions.
MeshDecimator: The mesh decimator was intended to reduce the amount of trianges in generated meshes but it has always had problems. It's far too slow and not very robust. For cubic mehes it is no longer needed anyway as the CubicSurfaceExtractor has built in decimation which is much more effective. For smooth (marching cubes) meshes there is no single alternative. You can consider downsampling the volume before you run the surface extractor (see the SmoothLODExample), and in the future we may add support for an external mesh processing library.
MeshDecimator: The mesh decimator was intended to reduce the amount of triangles in generated meshes but it has always had problems. It's far too slow and not very robust. For cubic meshes it is no longer needed anyway as the CubicSurfaceExtractor has built in decimation which is much more effective. For smooth (marching cubes) meshes there is no single alternative. You can consider downsampling the volume before you run the surface extractor (see the SmoothLODExample), and in the future we may add support for an external mesh processing library.
SimpleInterface: This was added so that people could create volumes without knowing about templates, and also to provide a target which was easier to wrap with SWIG. But overall we'd rather focus on geting the real SWIG bindings to work. The SimpleInterface is also extreamly limited in the functionality it provides.
SimpleInterface: This was added so that people could create volumes without knowing about templates, and also to provide a target which was easier to wrap with SWIG. But overall we'd rather focus on getting the real SWIG bindings to work. The SimpleInterface is also extremely limited in the functionality it provides.
Serialisation: This is a difficult one. The current serialisation is rather old and only deals with loading/saving a whole volume at a time. It would make much more sense if it could serialise regions instead of whole volumes, and if it could handle compression. It would then be useful for serialising the data which is paged into and out of the LargeVolume, as well as other uses. Both these changes would likely require breaking the current format and interface. It is likely that serialisation will return to PolyVox in the future, but for now we would suggest just implementing your own.
@ -18,13 +18,13 @@ Refactor of basic voxel types
-----------------------------
Previous versions of PolyVox provided classes representing voxel data, such as MaterialDensityPair88. These classes were required to expose certain functions such as getDensity() and getMaterial(). This is no longer the case as now even primitive data types such as floats and ints can be used as voxels. You can also create new classes to represent voxel data and it is entirely up to you what kind of properties they have.
As an example, imagine you might want to store lighting values in a volume and propergate them according to some algorithm you devise. In this case the voxel data has no concept of densities or materials and there is no need to provide them. Algorithms which conceptually operate on densities (such as the MarchingCubesSurfaceExtractor) will not be able to operate on your lighting data but this would not make sense anyway.
As an example, imagine you might want to store lighting values in a volume and propagate them according to some algorithm you devise. In this case the voxel data has no concept of densities or materials and there is no need to provide them. Algorithms which conceptually operate on densities (such as the MarchingCubesSurfaceExtractor) will not be able to operate on your lighting data but this would not make sense anyway.
Because algorithms now know nothing about the structure of the underlying voxels, some utility functions/classes are required. The principle is similar to the std::sort algorithm, which knows nothing about the type of data it is operating on but is told how to compare any two values via a comparator function object. In this sense, it will be useful to have an understanding of how algorithms such as std::sort are used.
We have continued to provide the Density, Material, and MaterialDensityPair classes to ease the transition into the new system, but really they should just be considered as examples of how you might create your own voxel types.
Some algothms assume that basic mathematical operations can be applied to voxel types. For example, the LowPassFilter needs to compute the average of a set of voxels, and to do this it needs to be possible to add voxels together and divide by an integer. Obviously these operations are provided by primitive types, but it means that if you want to use the LowPassfilter on custom voxel types then these types need to provide operator+=, operator/=, etc. These have been added to the Density and MaterialDensityPair classes, but not to the Material class. This reflects the fact that applying a low pass filter to a material volume does not make conceptual sense.
Some algorithms assume that basic mathematical operations can be applied to voxel types. For example, the LowPassFilter needs to compute the average of a set of voxels, and to do this it needs to be possible to add voxels together and divide by an integer. Obviously these operations are provided by primitive types, but it means that if you want to use the LowPassfilter on custom voxel types then these types need to provide operator+=, operator/=, etc. These have been added to the Density and MaterialDensityPair classes, but not to the Material class. This reflects the fact that applying a low pass filter to a material volume does not make conceptual sense.
Changes to build system
-----------------------
@ -40,13 +40,13 @@ Changes to Raycast
------------------
The raycasting functionality was previously in a class (Raycast) but now it is provided by standalone functions. This is basically because there is no need for it to be a class, and it complicated usage. The new functions are called 'raycastWithDirection' and 'raycastWithEndpoints' which helps remove the previous ambiguity regarding the meaning of the two vectors which are passed as parameters.
The callback functionalilty (called for each processed voxel to determine whether to continue and also perform additional processing) is no longer implemented as an std::function. Instead the standard STL approach is used, in which a function or function object is used a a template parameter. This is faster than the std::function solution and should also be easier to integrate with other languages.
The callback functionality (called for each processed voxel to determine whether to continue and also perform additional processing) is no longer implemented as an std::function. Instead the standard STL approach is used, in which a function or function object is used a a template parameter. This is faster than the std::function solution and should also be easier to integrate with other languages.
Usage of the new raycasting is demonstrated by a unit test.
Changes to AmbientOcclusionCalculator
-------------------------------------
The AmbientOcclusionCalculator has also been unclassed and is now called calculateAmbientOcclusion. The unit test has been updated to demsonstrate the new usage.
The AmbientOcclusionCalculator has also been unclassed and is now called calculateAmbientOcclusion. The unit test has been updated to demonstrate the new usage.
Changes to A* pathfinder
------------------------
@ -54,4 +54,4 @@ The A* pathfinder has always had different (but equally valid) results when buil
Copy and move semantics
-----------------------
All volume classes now have protecected copy constructors and assignment operators to prevent you from accidently copying them (which is expensive). Look at the VolumeResampler if you really do want to copy some volume data.
All volume classes now have protected copy constructors and assignment operators to prevent you from accidentally copying them (which is expensive). Look at the VolumeResampler if you really do want to copy some volume data.

18
documentation/LevelOfDetail.rst
View File

@ -1,15 +1,15 @@
***************
Level of Detail
***************
When the PolyVox surface extractors are applied to volume data the resulting mesh can contain a very high number of triangles. For large voxel worlds this can cause both performance and memory problems. The performance problems occur due the the load on the vertex shader which has to process a large number of vertices, and also due to the setup costs of a large number of tiny (possibly sub-pixel) triangles. The memory costs result simply from having a large amount of data which does not actually contibute to the visual appearance of the scene.
When the PolyVox surface extractors are applied to volume data the resulting mesh can contain a very high number of triangles. For large voxel worlds this can cause both performance and memory problems. The performance problems occur due the the load on the vertex shader which has to process a large number of vertices, and also due to the setup costs of a large number of tiny (possibly sub-pixel) triangles. The memory costs result simply from having a large amount of data which does not actually contribute to the visual appearance of the scene.
For these reasons it is desirable to reduce the triangle count of the meshes as far as possible, espessially as meshes move away from the camera. This document describes the various approaches which are available within PolyVox to achieve this. Generally these approches are different for cubic meshes vs smooth meshes and so we address these cases seperatly.
For these reasons it is desirable to reduce the triangle count of the meshes as far as possible, especially as meshes move away from the camera. This document describes the various approaches which are available within PolyVox to achieve this. Generally these approaches are different for cubic meshes vs smooth meshes and so we address these cases separately.
Cubic Meshes
============
A naive implementation of a cubic surface extractor would generate a mesh containing a quad for voxel face which lies on the boundary between a solid and an empty voxel. The CubicSurfaceExtractor is indeed capable of generating such a mesh, but it also provides the option to merge adjacent quads into a single quad subject to various conditions being satisfied (e.g. the faces must have the same material). This meging process drastically reduces the amount of geometry which must be drawn but does not modify the shape of the mesh. Because of these desirable properties such merging is performed by default, but it can be disabled if necessary.
A naive implementation of a cubic surface extractor would generate a mesh containing a quad for voxel face which lies on the boundary between a solid and an empty voxel. The CubicSurfaceExtractor is indeed capable of generating such a mesh, but it also provides the option to merge adjacent quads into a single quad subject to various conditions being satisfied (e.g. the faces must have the same material). This merging process drastically reduces the amount of geometry which must be drawn but does not modify the shape of the mesh. Because of these desirable properties such merging is performed by default, but it can be disabled if necessary.
To our knowledge the only drawback of performing this quad marging is that it can create T-junctions in the resulting mesh. T-junctions are an undesirable property of mesh geometry because they can cause tiny cracks (usually just seen as flickering pixels) to occur between quads. The figure below shows a mesh before quad merging, a mesh where the merged quads have caused T-junctions, and how the resulting rendering might look (note the single pixel holes along the quad border).
To our knowledge the only drawback of performing this quad merging is that it can create T-junctions in the resulting mesh. T-junctions are an undesirable property of mesh geometry because they can cause tiny cracks (usually just seen as flickering pixels) to occur between quads. The figure below shows a mesh before quad merging, a mesh where the merged quads have caused T-junctions, and how the resulting rendering might look (note the single pixel holes along the quad border).
*Add figure here...*
@ -19,19 +19,19 @@ Vertices C and D are supposed to lie exactly along the line which has A and B as
Whether it's a problem in practice depends on hardware precision (16/32 bit), distance from origin, number of transforms which are applied, and probably a number of other factors. We have yet to investigate.
We don't currently have a real solution to this problem. In Voxeliens the borders between voxels were darkened to simulate ambient occlusion and this had the desirable side effect of making any flickering pixels very hard to see. It's also possible that antialiasing stratagies can reduce the problem, and storing vertex positions as integers may help as well. Lastly, it may be possible to construct some kind of postprocess which would repair the image where it identifies single pixel discontinuities in the depth buffer.
We don't currently have a real solution to this problem. In Voxeliens the borders between voxels were darkened to simulate ambient occlusion and this had the desirable side effect of making any flickering pixels very hard to see. It's also possible that anti-aliasing strategies can reduce the problem, and storing vertex positions as integers may help as well. Lastly, it may be possible to construct some kind of post-process which would repair the image where it identifies single pixel discontinuities in the depth buffer.
Smooth Meshes
=============
Level of detail for smooth meshes is a lot more complex than for cubic ones, and we'll admit upfront that we do not currently have a good solution to this problem. None the less, we do have a couple of partial solutions which you might be able to use or adapt for your specific scenario.
Techniques for performing level of detail on smooth meshes basically fall into two catagories. The first category involves reducing the resolution of the volume data and then running the surface extractor on the smaller volume. This naturally generates a lower detail mesh which must then be scaled up to match the other meshes in the scene. The second category involves generating the mesh at full detail and then using traditional mesh simplification techniques to reduces the number of trianges. Both techniques are explored in more detail below.
Techniques for performing level of detail on smooth meshes basically fall into two categories. The first category involves reducing the resolution of the volume data and then running the surface extractor on the smaller volume. This naturally generates a lower detail mesh which must then be scaled up to match the other meshes in the scene. The second category involves generating the mesh at full detail and then using traditional mesh simplification techniques to reduces the number of triangles. Both techniques are explored in more detail below.
Volume Reduction
----------------
The VolumeResampler class can be used to copy volume data from a source region to a destination region, and it handles the resampling of the voxel values in the event that the source and destination regions are not the same size. This is exactly what we need for implementing level of detail and the principle is demonstrated by the SmoothLOD sample (see the documentation for the SmoothLOD sample for more information).
One of the problems with this approach is that the lower resolution mesh does not *exactly* line up with the higher resolution mesh, and this can caus cracks to be visible where the two meshes meet. The SmoothLOD sample attempts to avoid this problem by overlapping the meshes slightly but this may not be effective in all situations or from all viewpoints.
One of the problems with this approach is that the lower resolution mesh does not *exactly* line up with the higher resolution mesh, and this can cause cracks to be visible where the two meshes meet. The SmoothLOD sample attempts to avoid this problem by overlapping the meshes slightly but this may not be effective in all situations or from all viewpoints.
An alternative is the Transvoxel algorithm (link) developed by Eric Lengyel. This essentially extends the original Marching Cubes lookup table with additional entries which handle seamless transitions between LOD levels, and it is a very promising solution to level of detail for voxel terrain. At this point in time we do not have an implementation of this algorithm but work is being undertaking in the area. For the latest developments see: http://www.volumesoffun.com/phpBB3/viewtopic.php?f=2&t=338
@ -43,6 +43,6 @@ The other main approach is to generate the mesh at the full resolution and then
The first additional complication is that the decimation algorithm needs to preserve material boundaries so that they don't move between LOD levels. When choosing whether a particular simplification can be made (i.e deciding if one vertex can be collapsed on to another or whether two faces can be merged) a metric is usually used to determine how much the simplification would affect the visual appearance of the mesh. When working with smooth voxel meshes this metric needs to also consider the material identifiers.
We also need to ensure that the metric preserves the geometric boundary of the mesh, so that no cracks are visible when a simplified mesh is place next to an original one. Maintaining this geometric boudary can be difficult, as the straightforward approach of locking the edge vertices in place will tend to limit the amount of simplification which can be performed. Alternatively, cracks can be allowed to develop if they are later hidden through the use of 'skirts' around the resulting mesh.
We also need to ensure that the metric preserves the geometric boundary of the mesh, so that no cracks are visible when a simplified mesh is place next to an original one. Maintaining this geometric boundary can be difficult, as the straightforward approach of locking the edge vertices in place will tend to limit the amount of simplification which can be performed. Alternatively, cracks can be allowed to develop if they are later hidden through the use of 'skirts' around the resulting mesh.
PolyVox used to contain code for performing simplification of the smooth voxel meshes, but unfortuanatly it had significant performance and functionality issues. Therefore it has been deprecated and it will be removed in a future version of the library. We will instead investigate the use of external mesh simplification libraries and OpenMesh (link) may be a good candidate here.
PolyVox used to contain code for performing simplification of the smooth voxel meshes, but unfortunately it had significant performance and functionality issues. Therefore it has been deprecated and it will be removed in a future version of the library. We will instead investigate the use of external mesh simplification libraries and OpenMesh (link) may be a good candidate here.

14
documentation/Lighting.rst
View File

@ -1,23 +1,23 @@
********
Lighting
********
Lighting is an important part of creating a realistic scene, and fortunatly most common lighting solutions can be easily applied to PolyVox meshes. In this document we describe how to implement dynamic lighting and ambient occlusion with PolyVox.
Lighting is an important part of creating a realistic scene, and fortunately most common lighting solutions can be easily applied to PolyVox meshes. In this document we describe how to implement dynamic lighting and ambient occlusion with PolyVox.
Dynamic Lighting
================
In general, any lighting solution for realtime 3D graphics should be directly applicable to PolyVox meshes.
In general, any lighting solution for real-time 3D graphics should be directly applicable to PolyVox meshes.
Normal Calculation for Smooth Meshes
------------------------------------
When working with smooth voxel terrain meshes, PolyVox provides vertex normals as part of the extracted surface mesh. A common approach for computing these normals would be to compute normals for each face in the mesh, and then compute the vertex normals as a weighted average of the normals of the faces which share it. Actually this is not the approach used by PolyVox, as PolyVox instead computes the vertex normals directly from the underlying volume data.
More specifically, PolyVox is able to compute the *gradient* of the volume data at any given point using well established image processing methods. The normalised gradient value is used as the vertex normal and in general it is smoother than the value computed by averaging neighbouring faces. Actually there are two approaches to this gradient comoputation known as *Central Differencing* and *Sobel Filter*. The central differencing approach is generally recommended but the Sobel filter can be used to obtain slightly smoother results but with lower performance. See the MarchingCubesSurfaceExtractor documentation for details on how to select between these (check this exists...).
More specifically, PolyVox is able to compute the *gradient* of the volume data at any given point using well established image processing methods. The normalised gradient value is used as the vertex normal and in general it is smoother than the value computed by averaging neighbouring faces. Actually there are two approaches to this gradient computation known as *Central Differencing* and *Sobel Filter*. The central differencing approach is generally recommended but the Sobel filter can be used to obtain slightly smoother results but with lower performance. See the MarchingCubesSurfaceExtractor documentation for details on how to select between these (check this exists...).
Normal Calculation for Cubic Meshes
-----------------------------------
For cubic meshes PolyVox doesn't actually generate any vetex normals at all, and this is often a source of confusion for new users. The reason for this is that we wish to to perform per-face lighting rather than per-vertex lighting. Considering the case of a single cube, if we wanted to perform per-face lighting based on per-vertex normals then the normals cannot be shared between adjacent faces and so each vertex needs to be duplicated three times (one for each face which uses it). This means we would need 24 vertices to represent a cube which intuitively should only need eight vertices.
For cubic meshes PolyVox doesn't actually generate any vertex normals at all, and this is often a source of confusion for new users. The reason for this is that we wish to to perform per-face lighting rather than per-vertex lighting. Considering the case of a single cube, if we wanted to perform per-face lighting based on per-vertex normals then the normals cannot be shared between adjacent faces and so each vertex needs to be duplicated three times (one for each face which uses it). This means we would need 24 vertices to represent a cube which intuitively should only need eight vertices.
Therefore PolyVox does not generate per-vertex normals for cubic meshes, and as a result the cubic mesh's vertices are both smaller and less numerous. Of course, we still need a way to obtain normals for lighting calcualtions and so our suggestion is to compute the normals in a fragment program using the *derivative operations* which are provided by modern graphics hardware.
Therefore PolyVox does not generate per-vertex normals for cubic meshes, and as a result the cubic mesh's vertices are both smaller and less numerous. Of course, we still need a way to obtain normals for lighting calculations and so our suggestion is to compute the normals in a fragment program using the *derivative operations* which are provided by modern graphics hardware.
The description here is rather oversimplified, but the idea behind these operation is that they can tell you how much a variable has changed between two adjacent pixels. If we use our fragment world space position as the input to these derivative operations then we can obtain two vectors which lie on the surface of our face. The cross product of these then gives us a vector which is perpendicular to both and which is therefore our normal.
@ -34,9 +34,9 @@ Similar code can be implemented in HLSL but you may need to invert the normal du
Shadows
-------
To date we have only experimented with shadow maps as a solution to the real time shadowing problem and have found they work very well for both casting and receiving. The approach is essentially the same as for any other geometry and the usual approches can be used for setting up the projection and for filtering the result. One PolyVox specific tip we can give is that you don't need to take account of materials when rendering into the shadow map as you always draw in black, so if you are splitting you geometry for material blending or for handling a large number of materials then you don't need to do this when rendering shadows. Using sepeate shadow geometry with all materials combined may decrease your batch count in this case.
To date we have only experimented with shadow maps as a solution to the real time shadowing problem and have found they work very well for both casting and receiving. The approach is essentially the same as for any other geometry and the usual approaches can be used for setting up the projection and for filtering the result. One PolyVox specific tip we can give is that you don't need to take account of materials when rendering into the shadow map as you always draw in black, so if you are splitting you geometry for material blending or for handling a large number of materials then you don't need to do this when rendering shadows. Using separate shadow geometry with all materials combined may decrease your batch count in this case.
The most widely used alternative to shadow maps is shadow volumes but we have not tested these with PolyVox. We do not expect these will provide a good solution because meshes usually require additional edge information to allow the shadaw volume to be extruded from the silhouette and PolyVox does not provide this. Even if this edge information could be calculated, it would be invalidated each time the mesh changed which would make dynamic terrain more difficult.
The most widely used alternative to shadow maps is shadow volumes but we have not tested these with PolyVox. We do not expect these will provide a good solution because meshes usually require additional edge information to allow the shadow volume to be extruded from the silhouette and PolyVox does not provide this. Even if this edge information could be calculated, it would be invalidated each time the mesh changed which would make dynamic terrain more difficult.
Overall we would recommend you make use of shadow maps for dynamic shadows.

12
documentation/Prerequisites.rst
View File

@ -3,7 +3,7 @@ Prerequisites
*************
The PolyVox library is aimed at experienced games and graphics programmers who wish to incorporate voxel environments into their applications. It is not a drop in solution, but instead provides a set of building blocks which you can use to construct a complete system. Most of the functionality could be considered quite low-level. For example, it provides mesh extractors to generate a surface from volume data but requires the application developer to handle all tasks related to scene management and rendering of such data.
As a result you will need a decent amount of graphics programming experience to effectively make use of the library. The purpose of this document is to highlight some of the core areas with which you will need to be familier. In some cases we also provide links to places where you can find more information about the subject in question.
As a result you will need a decent amount of graphics programming experience to effectively make use of the library. The purpose of this document is to highlight some of the core areas with which you will need to be familiar. In some cases we also provide links to places where you can find more information about the subject in question.
You should also be aware that voxel terrain is still an open research area and has not yet seen widespread adoption in games and simulations. There are many questions to which we do not currently know the best answer and so you may have to do some research and experimentation yourself when trying to obtain your desired result. Please do let us know if you come up with a trick or technique which you think could benefit other users.
@ -11,9 +11,9 @@ Programming
===========
This section describes some of the programming concepts with which you will need to be familiar:
**C++:** PolyVox is written using the C++ language and we expect this is what the majority of our users will be developing in. You will need to be familer with the basic process of building and linking against external libraires as well as setting up your development environment. Note that you do have the option of working with other languages via the SWIG bindings but you may not have as much flexibility with this approach.
**C++:** PolyVox is written using the C++ language and we expect this is what the majority of our users will be developing in. You will need to be familiar with the basic process of building and linking against external libraries as well as setting up your development environment. Note that you do have the option of working with other languages via the SWIG bindings but you may not have as much flexibility with this approach.
**Templates:** PolyVox also makes heavy use of template programming in order to be both fast and generic, so familiarity with templates will be very useful. You should't need to do much template programming yourself but an understanding of them will help you understand errors and resolve any problems.
**Templates:** PolyVox also makes heavy use of template programming in order to be both fast and generic, so familiarity with templates will be very useful. You shouldn't need to do much template programming yourself but an understanding of them will help you understand errors and resolve any problems.
**Callbacks:** Several of the algorithms in PolyVox can be customised through the use of callbacks. In general there are sensible defaults provided, but for maximum control you may wish to learn how to define you own. In general the principle is similar to the way in which callbacks are used in the STL.
@ -21,7 +21,7 @@ Graphics Concepts
=================
Several core graphics principles will be useful in understanding and using PolyVox:
**Volume representation:** PolyVox revolves around the idea of storing and manipulating volume data and using this as a representation of a 3d world. This is a fairly intuitaive extension of traditional heightmap terrain but does require the ability to 'think in 3D'. You will need to understand that data stored in this way can become very large, and understand the idea that paging and compression can be used to combat this.
**Volume representation:** PolyVox revolves around the idea of storing and manipulating volume data and using this as a representation of a 3d world. This is a fairly intuitive extension of traditional heightmap terrain but does require the ability to 'think in 3D'. You will need to understand that data stored in this way can become very large, and understand the idea that paging and compression can be used to combat this.
**Mesh representation:** Most PolyVox projects will involve using one of the surface extractors, which output their data as index and vertex buffers. Therefore you will need to understand this representation in order to pass the data to your rendering engine or in order to perform further modifications to it. You can find out about more about this here: (ADD LINK)
@ -29,9 +29,9 @@ Several core graphics principles will be useful in understanding and using PolyV
Rendering
=========
**Runtime geometry creation:** PolyVox is independant of any particular graphics API which means it outputs its data using API-neutral structures such as index and vertex buffers (as mentioned above). You will need to write the code which converts these structures into a format which your API or engine can understand. This is not a difficult task but does require some knowledge of the rendering technology which you are using.
**Runtime geometry creation:** PolyVox is independent of any particular graphics API which means it outputs its data using API-neutral structures such as index and vertex buffers (as mentioned above). You will need to write the code which converts these structures into a format which your API or engine can understand. This is not a difficult task but does require some knowledge of the rendering technology which you are using.
**Scene management:** PolyVox is only responsible for providing you with the mesh data to be displayed, so you application or engine will need to make sensible decisions about how this data should be organised in terms of a spactial structure (octree, bounding volumes tree, etc). It will also need to provide some approach to visibility determination such as frustum culling or a more advanced appraoch. If you are integrating Polyvox with an existing rendering engine then you should find that many of these aspects are already handled for you.
**Scene management:** PolyVox is only responsible for providing you with the mesh data to be displayed, so you application or engine will need to make sensible decisions about how this data should be organised in terms of a spatial structure (octree, bounding volumes tree, etc). It will also need to provide some approach to visibility determination such as frustum culling or a more advanced approach. If you are integrating PolyVox with an existing rendering engine then you should find that many of these aspects are already handled for you.
**Shader programming:** The meshes which are generated by PolyVox are very basic in terms of the vertex data they provide. You get vertex positions, a material identifier, and sometimes vertex normals. It is the responsibility of application programmer to decide how to use this data to create visually interesting renderings. This means you will almost certainly want to make use of shader programs. Of course, in our texturing (LINK) and lighting (LINK) documents you will find many ideas and common solutions, but you will need strong shader programming experience to make effective use of these.

24
documentation/Threading.rst
View File

@ -3,9 +3,9 @@ Threading
*********
Modern computing hardware typically contains a large number of processors, and so users of PolyVox often want to know how they can best make use of these from their applications.
PolyVox does not make any guarentees about thread-saftey, and does not contain any threading primitives to protect access to data structures. You can still make use of PolyVox from multiple threads, but you will have to take responisbility for enforcing thread saftey yourself (e.g. by providing thread safe wrappers around the volume classes). If you do want to use PolyVox is a multi-threaded context then this document provides some tips and tricks that you might find useful.
PolyVox does not make any guarantees about thread-safety, and does not contain any threading primitives to protect access to data structures. You can still make use of PolyVox from multiple threads, but you will have to take responsibility for enforcing thread safety yourself (e.g. by providing thread safe wrappers around the volume classes). If you do want to use PolyVox is a multi-threaded context then this document provides some tips and tricks that you might find useful.
However, be aware that we do not have a lot of expertise in threading, and this is part of the reason why it is not explicitely addressed within PolyVox. If you do have more experience and believe any of this information to be misleading then please do post on the forums to discuss it.
However, be aware that we do not have a lot of expertise in threading, and this is part of the reason why it is not explicitly addressed within PolyVox. If you do have more experience and believe any of this information to be misleading then please do post on the forums to discuss it.
Volumes
=======
@ -15,21 +15,21 @@ RawVolume
---------
The RawVolume has a very simple internal structure in which the data is stored as a single array which is never moved or resized. Because of this property it is indeed safe to perform multiple simultaneous *reads* of the data from different threads. However, it is generally not safe to perform simultaneous writes from different threads, or even to write from only one thread while other threads are reading.
The reason why simultaneous writres can be problematic should be fairly obvious - if two different threads try to write to the same voxel then the result will depend on which thread writes first. But why can't we write from one thread and read from another one? The problem here is that the the CPU may implement some kind of caching mechanism and/or choose to store data in registers. If a write operation occurs before a read, then the read *may* still obtain the old value because the cache has not been updated yet. There may even be a cache for each thread (particularly if running on multiple processors).
The reason why simultaneous writes can be problematic should be fairly obvious - if two different threads try to write to the same voxel then the result will depend on which thread writes first. But why can't we write from one thread and read from another one? The problem here is that the the CPU may implement some kind of caching mechanism and/or choose to store data in registers. If a write operation occurs before a read, then the read *may* still obtain the old value because the cache has not been updated yet. There may even be a cache for each thread (particularly if running on multiple processors).
If we assume for a moment that each voxel is a simple integer, then the rules for accessing a single voxel from multiple threads are the same as the rules for accessing integers from multiple threads. There is some useful information about this available on the web, including a discussion on StackOverflow: http://stackoverflow.com/questions/4588915/can-an-integer-be-shared-between-threads-safely
If nothing else, this serves to illustrate that multi-threaded access even to something as simple as an integer can be suprisingly complex annd architecture dependant.
If nothing else, this serves to illustrate that multi-threaded access even to something as simple as an integer can be surprisingly complex and architecture dependant.
However, all this has been in the context of accessing a *single* voxel from multiple threads... what if we carefully design our algotithm such that diferent threads access different parts of the volume which never overlap? Unfortunatly I don't believe this is a solution either. Caching mehanisms seldom operate on indiviual elements but instead tend to cache larger chunks of data on the grounds that data accesses are usually localised. So it's quite possible that accessing a given voxel will cause another voxel to be cached.
However, all this has been in the context of accessing a *single* voxel from multiple threads... what if we carefully design our algorithm such that different threads access different parts of the volume which never overlap? Unfortunately I don't believe this is a solution either. Caching mechanisms seldom operate on individual elements but instead tend to cache larger chunks of data on the grounds that data accesses are usually localised. So it's quite possible that accessing a given voxel will cause another voxel to be cached.
C++ does provide the 'volatile' keyword which can be used to ensure a variable is updated immediatly (rather than being cached) but this is still not sufficient for thread safe code. It also has performance implications which we would like to avoid. More information about volatile and multitheaded programming can be found here: http://software.intel.com/en-us/blogs/2007/11/30/volatile-almost-useless-for-multi-threaded-programming/
C++ does provide the 'volatile' keyword which can be used to ensure a variable is updated immediately (rather than being cached) but this is still not sufficient for thread safe code. It also has performance implications which we would like to avoid. More information about volatile and multitheaded programming can be found here: http://software.intel.com/en-us/blogs/2007/11/30/volatile-almost-useless-for-multi-threaded-programming/
Lastly, note that PolyVox volumes are templatised which means the voxel type might be something other than a simple int. However we don't think this actually makes a difference given that so few guarentees are made anyway, and it should still be safe to perform multiple concurrent reads for more complex types.
Lastly, note that PolyVox volumes are templatised which means the voxel type might be something other than a simple int. However we don't think this actually makes a difference given that so few guarantees are made anyway, and it should still be safe to perform multiple concurrent reads for more complex types.
LargeVolume
-----------
The LargeVolume provides even less thread safety than the RawVolume, in that even concurrent read operations can cause problems. The reason for this is the more complex memory management which is performed behind the scenes, and which allows peices of volume data to be moved around and deleted. For example, a read of a single voxel may mean that the block of data associated with that voxel has to be paged in to mamory, which in turn may mean that another block of data has to be paged out of memory. If second thread was halfway through reading a voxel in this second block of data then a problem will occur.
The LargeVolume provides even less thread safety than the RawVolume, in that even concurrent read operations can cause problems. The reason for this is the more complex memory management which is performed behind the scenes, and which allows pieces of volume data to be moved around and deleted. For example, a read of a single voxel may mean that the block of data associated with that voxel has to be paged in to memory, which in turn may mean that another block of data has to be paged out of memory. If second thread was halfway through reading a voxel in this second block of data then a problem will occur.
In the future we may do a more comprehensive analysis of thread safety in the LargeVolume, but for now you should assume that any multithreaded access can cause problems.
@ -37,7 +37,7 @@ Consequences of abuse
---------------------
We have outlined above the rules for multithreaded access of volumes, but what actually happens if you violate these? There's a couple of things to watch out for:
- As mentioned, performing unprotected writes to the volume can cause problems because the data may be copied into the CPU cache and/or registers, and so a subsequent read could retrieve thoe old value. This is not what you want but probably won't be fatal (i.e. it shouldn't crash). It would basically manifest itself as data corruption.
- As mentioned, performing unprotected writes to the volume can cause problems because the data may be copied into the CPU cache and/or registers, and so a subsequent read could retrieve the old value. This is not what you want but probably won't be fatal (i.e. it shouldn't crash). It would basically manifest itself as data corruption.
- If you access the LargeVolume in a multithreaded fashion then you risk trying to access data which has been removed by another thread, and in this case you will get undefined behaviour. This will probably be a crash (out of bounds access) but really anything could happen.
Surface Extraction
@ -46,11 +46,11 @@ Despite the lack of thread safety built in to PolyVox, it is still possible and
Combining multiple surface extraction threads with the *LargeVolume* is something we will need to experiment with in the future, to determine how it can be improved.
In the future we will expand this section to discuss how to split surace extraction across a number of threads, but for now please see Section XX of the book chapter 'Volumetric Representation of Virtual environments', available for free here: http://books.google.nl/books?id=WNfD2u8nIlIC&lpg=PR1&dq=game+engine+gems&pg=PA39&redir_esc=y#v=onepage&q&f=false
In the future we will expand this section to discuss how to split surface extraction across a number of threads, but for now please see Section XX of the book chapter 'Volumetric Representation of Virtual environments', available for free here: http://books.google.nl/books?id=WNfD2u8nIlIC&lpg=PR1&dq=game+engine+gems&pg=PA39&redir_esc=y#v=onepage&q&f=false
GPU thread safety
=================
Be aware that even if you sucessfully perform surface across multiple threads you still need to take care when uploading the data to the GPU. For Direct3D 9.0 and OpenGL 2.0 it is only possible to upload data from the main thread (or more accuratly the one which owns the rendering context). So after you have performed your multi-threaded surface extraction you need to bring the data back to the main thread for uploading to the GPU.
Be aware that even if you successfully perform surface across multiple threads you still need to take care when uploading the data to the GPU. For Direct3D 9.0 and OpenGL 2.0 it is only possible to upload data from the main thread (or more accurately the one which owns the rendering context). So after you have performed your multi-threaded surface extraction you need to bring the data back to the main thread for uploading to the GPU.
More recent versions of the Direct3D and OpenGL APIs lift this restriction and provide means of accessing GPU resources from multiple threads. Please consult the documentation for your API for details.
@ -64,4 +64,4 @@ It might be useful to provide a thread safe wrapper around the volume classes, a
OpenMP
------
This is a standard for extending C++ with compiler directives which allow the compiler to automatically parallise sections of code. Most likely this could be used to parallelise some of the loops which occur in image processing tasks.
This is a standard for extending C++ with compiler directives which allow the compiler to automatically parallelise sections of code. Most likely this could be used to parallelise some of the loops which occur in image processing tasks.