Engine Update & New Coding Platform

Hello Friends, 

A while it has been since we last wrote one of these so called blogs and today, we wish to share with you the progress we have made so far regarding the engine and its status. 
As you all know, it has been a tough year for many with this virus at hand and also to suffer major bereavement just makes it unbearable at times, but as we always have said, we try to maintain development as best we can even if that means we dont work on the engine everyday since we have to have immediate earnings over long term investments.

Our faith in the engine has not changed, we still believe what we have is unique and fun / playful for every developer of all ranges and ages and strive to achieve our goal regardless of circumstance.

So what has changed, engine wise not too much apart from PhysX 3.1 upgrades, GPU streamed textures which allows faster loading of worlds / scenes and also the shader upgrade to maintain industry standard PBR. For us it is important that we have the pipeline to directly export assets from major toolsets and they look the same inside skyline, for instance, exporting PBR from Substance Painter, Quixel, and other top end tools. However we have had to take time aside to focus on the Scripting Pipeline which was sorely suffering from some backdated code aswell as being integrated directly into the engine which was never good if the engine crashed as you would lose all the script code you wrote unless saved and when loading you would have to find all the files you were working on.

So this gives you the idea of this bit of news / blog on the progress of the engine which has taken some time, but now the script editor is as good as the rest and could quite easily replace many other scripting pipelines you use in your daily works.

Script Editor

So to start lets have a brief overview of the changes to the editor and then go through each section explaining the changes:

Script Editor Changes

• Now a seperate application which is Aurasoft UK's second public application that will be downloadable from the store with its own installer.
• Complete rewrite of the entire interface and workflow yet remaining the same for existing users.
• Project & Workspace Management. More matching todays current applications like VSCode and Atom Editor.
• Custom Lexers and colouring for all Filetypes including custom filetypes
• Fixed and improved extensible Intellisense and calltips
• Tabs updates including context menus and Tab Colouring
• Editor remembers each workspaces last working files, stores current file scripts and restores when reopened.
• Split screen editor for 2 concurrent files
• Improved Find and replace
• Improved Page editing
• All lexer / API / Snippets from hard drive rather than hard coded
• Faster loading than even notepad++ but more features.
• Direct hooking from Script Editor to Skyline Game Engine for Microscripts and other editor functionality.
• Used as the in house script / text editor for all programming needs.

Why the change?

Lets start off with the first set of points, the reason why we made this a separate application and why its taken the amount of time it has.
We have been avid VSCode users with its many possibilities to do what we want how we want but as it would seem with all electron based applications, they are bloated and tiresome to use due to their size. Not everyone has the most up to date CPU and lets face it, scripting or coding is a main part of any game engines development pipeline unless its a stuck in a box, do what you have type engine, which we all know skyline is not. Which is why we have taken the effort to make an extensible seperate application apart from the engine that is both fast and efficient when it comes to scripting or coding or even text editing. But because you may not always use it with the game engine in mind, this has led to many new features and ways of working while also being a direct tool to interface with the engine.

So the first point is, how does this work with the engine then:
Well simple, Windows messaging system allows us to communicate directly between two applications or more through c++, which we have extended to make the script editor a viable usable lua tool while working with the engine. When you open a Microscript or scene script in the editor; it is immediately sent to the seperate application of the script editor and when saved, sends the data back to the game engine. But many use cases came up. What if the files you have open are from a different scene or different set of entities in the game engine?, what if you have multiple versions of skyline game engine open with different scenes and different scene scripts / entity microdata open. The script editor has to know where to send the data and correctly set all the data back. This wasnt a problem with the older way of working as the script editor was opened with each instance of the game engine open, but this would mean several script editors for every open scene editor... clumsy and awkward to work with. Now we can have the one singular script editor and interface between each open game engine.

Because of the messaging system, it also means we can also directly control any skyline game engine editor directly without even touching the game editor... so if there are features like this you think would benefit the engine, then please let us know.

As we have said, we have developed this system to become the in-house editor for all our scripting and text editor needs without the need for any other package like notepad++ or vscode, atom editor, or any other package, because this applies; if you cant use your own tools, then your systems are not as good as you think. So this system replaces all needs of other software while at the same time gives a marketable advantage over other software. You should see within a few month the script editor released as its own tool and its own campaigns and bits starting to happen to make Aurasoft UK a proper brand, however as with all things, this requires adopters, feature enhancments and bug fixes along the way as its own piece of software. 
Here is a quick demonstration of this happening:

Workspaces:

One of the many aspects of VSCode we used so much was the project management, which we have integrated called workspaces. A workspace is a collection of root folders that allow you to quickly navigate files and folders that you need to work with. So for example, we could have a workspace called Skyline Scripts and in it, could be every content library scripts folder, or if you are developing shaders, then you could have a workspace of all your shader files without having cluttered or hard to reach file paths.

We do this using operating system symlinks that give complete access and control over a directory from a different location.
The workspaces also control what files and tabs are open as well as the last thing you edited. What this means is that you can edit a file without saving, close the script editor and reopen and continue programming where you left off aswell as keeping tab colours, tab orders and the ability to change worspace on the fly and close all tabs and open the new workspaces tabs and files and continue where you left off with those. (If you noticed, we mentioned tab colouring which was a major pain to implement but can really make it easier to know what file or type of file you are working on.)

Here is an example of the workspaces changing from one to the other:

Lexers and custom colouring:

However when it comes to working with files, all files except Lua in the previous versions of the script editor had basic one colour text which was white. Not very good when it comes to editing materials with JSON, or other files. So many built in languages have lexers straight away including the ability to write your own lexers for missing languages such as HLSL, GLSL or other file extensions. For instance, you might write your own file type with custom syntax and want it to be readable in the editor, with this system you can very easily as all extended lexers are controlled from the default library location which by default is "C:\Users\jayce\Documents\Aurasoft UK\Script Editor\Lexers\"

As mentioned with lexers, all Lexer / API and intellisense now comes directly from disk rather than hard coded internally. To extend the Lua language for Skyline Game Engine, we added the ability to load custom lexer data / information from different folders which means you can extend any language to add the necessary commands you project uses, in case of Skyline Game Engine, this means an entire new set of libraries and sub commands that are added to both the intellisense and calltips( Intellisense is the window which appears when you type entity.function() and calltips are what appear when entering the arguments for a function. )

As mentioned, we used this system to create a readable shader file with better colouring and with a file that has custom c++ directives which dont exist in HLSL to begin with. This means that the next set of shader jobs with the main engine will be programmed directly in the Script editor all users use, which i think is a pretty good sell of the product lol :P

Some pictures of the lexer bits and bobs:

Lexer Extensions file which controls which lexer file loads from what extension:

and the custom lexer file itself:

Intellisense / API usage

In the previous version of skyline, the script editor didnt have the nicest intellisence or calltips to which we have styled and upgraded a lot. With the ability to extend any languages known functions or intellisense in case you make something super custom.

Alongside all the cool worspace improvements, we didnt like the fact that we couldnt edit two files at the same time on a split screen, so we have implemented this system which makes it so cool now. As we can use 2 files especially with lua when pulling from include files etc.... 
This also is retrieved back when opening and closing the editor.... woop!!
The split screen seems to work flawlessly at this point and is also dependant on the workspaces used.

Other tweaks and changes:
Find and replace on files has also been improved as it was too clucky before, the UI for it has not changed, but now you can find and replace on any active file without having seperate windows open for each page you were working on. To say it was difficult to work with was an understatement and now it is much more similar to other editors in way the find and replace works.

File Page editing has also been upgraded giving Duplicate using crtl+D,  alt drag editing and more.

I think that covers the basic overall features that have been made for the script editor, which we hope comes across as a lot as it taken many weeks and a lot of programming to go from where it was to where it is now while not losing the feel of what we have had all along.

Some images:

Conclusion / Roadmap

With the script editor complete, this means we are now moving back to the development of the engine.
Since going v2.0 we have only had 1 developer( Me: Jayce / SolarPortal ) active since the changes required stopped the development of the second developer( hannah / Starfire ) from continuing with the Lua SDK requiring a major overhaul as the way it was implemented, stopped all the previous made scripts from working, this is a big no no when looking back as it would affect every game and developer that has used skyline so the plan is to leave the libraries as they are and start using a primary "sdk.library.function()" system instead which allows us to give you access to all the core aspects of the engine allowing complete control over rendering new windows in custom plugins, new nodes and meshes without the use of entities for lightweight execution or simply add functionality to the engine  which we did not have prior. I know we have talked about the Lua SDK extensively before so i wont go into it too much here.

Following on from changing the Lua SDK to be more backwards compatible, i will be going back into completing shaders which included terrain shaders and ocean shaders. Hannah / Starfire will be back into programming lua game based content that can be provided with the engine or sold on the store as remember future versions of skyline will be provided for free with commercial rights, so we have to make money from the engine other ways.

After the shader changes i will be moving onto making a server for skylines development so we can avoid the crisis of the V1 skyline pull that happened. This means that we will have a master branch stored elsewhere and a development branch on each of our own computer systems that when developing a feature we dont stop the release of a bug fix or something required quickly and always have an active branch. This may be done through private offline github or something similar. 
We said when we pulled the sold version that as a business we have to ensure that this mistake does not happen again as it can leave quite a hole to fill haha.. we are ashamed it happened and have learnt a valuable lesson since.

Once this system has been made for rapid active development on our end and be able to develop new features on a second branch on our own branches that eventually get merged into the master branch, then i will be moving onto the development upgrade of the terrain system as mentioned in this forum post: https://forum.aurasoft-skyline.co.uk/viewtopic.php?f=32&t=2607

Extremely good terrain editing has been apart of skyline since the start of the engine and we want to keep this rolling as well as a cool foliage editing system that will be provided as a seperate plugin which currently is the Skyline Eco System. Due to the changes in Lua SDK all features need to be updated to use the newer commands so there will be time on this.

Furthermore while mentioning plugins, the navmesh editor will go through its final tweaks as starfire will need the functions for the better characters and AI she has been developing, which due to my errors and mistakes stopped her moving forwards, but as mentioned above we are aiming to rectify this quickly.

And finally the other great news!!!! Vulkan Rendering System support which allows a cross compatible rendering library that is modern and uses the newer GPU architecture of accessing much more root that previous system like DX11 or OpenGL, the really good news about this is that Vulkan is Android Compatible which should mean that skyline will be able to be deployed to mobile platforms, but this may come after the release of V2.0. Speaking of mobile, with vulkan for android, i know the question is sitting there; well what about IOS? Well this is where the Metal Render system comes into play which is a cross platform between MacOS and IOS mobile(Apple A7 or newer) development. This will start to move skyline into a fully fledged engine which is what we envisioned for Gen2, but now seems more likely to happen on the upcoming Gen3.0 system.

As with all the development, our primary focus is bringing our second developer back in and getting a Beta out as soon as possible, this may mean not to different from the version you have, but the updated interface and rendering and keep this updated with the concurrent development of the engine of the remaining features :)

I dont think there is much more to talk about apart from we hope that you are all well, are staying safe from this horrible world changing virus and that you are all getting on well with your projects.
Wishing you all the best
Skyline Development Team!
:D