Mapping when exporting and importing for Web Applications (such as CSPs).

[GabsReDeal]

This PR allows mapping when exporting and importing for Web Applications (such as CSPs).

Currently, mapping is only implemented when exporting cls, mac, int, and inc files. This aims to allow mapping of web applications files (such as CSPs) when exporting and importing.

This aims to solve issues when working with multiple applications, but one source.

[GabsReDeal]

Once implemented, the documentation would need to be updated.

[isc-bsaviano]

Hi @GabsReDeal, thanks for the PR. Can you explain why you think this necessary?

[GabsReDeal]

Hello @isc-bsaviano

Currently working on integrating proper source control into our workflow and we have web files (CSP in our case) which are the same source in different web applications. Basically each developer will have their own workspace and web application on the server (which mimics working locally).

In our repository, we would like that all web files are in one folder (csp for instance), and not the name of the application. As having them in separate folder would not make sense.

The import was created to properly sync with the server when saving code.

[isc-bsaviano]

@GabsReDeal Can you include some example file paths and web app paths? I don't understand your workflow so it's hard for me to recommend an alternate approach.

[GabsReDeal]

This would be the scenario.

2 developers are working on one InterSystems Cache/IRIS server. Each developer has their own workspace to avoid conflicts with the other.

Developer one is on workspace cspdev1 and developer 2 is on cspdev2. The same would be for their web application and their files.

So the below would be developer 1's export and import map. Where the web files from /cspdev1/ are exported to the /csp/ (local repository), and when saving files in /csp/ are changed back to /cspdev1/ when syncing (and compiling) to the server.

"objectscript.export": {
    "map":{
        "\/(cspdev1)\/(.*)": "/csp/$1"
    }
},
"objectscript.import": {
    "map":{
        "\/csp\/(.*)": "/cspdev1/$1"
    }
}

in the above example if a user would to export /cspdev1/my/test.csp, it would save to src/csp/my/test.csp. Then when saving src/csp/my/test.csp, before importing it would convert back to /cspdev1/my/test.csp which will sync (and compile) successfully.

Feel free to let me know if there is a simpler solution where multiple developers are working on the same server but different workspaces.

[isc-bsaviano]

Thanks for the example, that's very helpful. I recommend that each developer has their own private IRIS instance for development. This is easy to set up now that we have Community Edition kits and containers. The developers can then have their own copy of the source code as well as an environment to develop and test it. In this model IRIS acts as an SDK, like the JVM or Node.js. This avoids the possibility of overwriting other devs' code on the shared instance, and eliminates the mental load of having to know that their web app path is different than what your end users will see.

[GabsReDeal]

Thanks for your quick replies.

While I agree that this would he the ideal scenario, we currently have a legacy system which has some obstacles.

We are currently still on Caché and planning to move to IRIS later this year. We were not going to have an individual local instance due to the database. We currently have a large database and we are not allowed to have such a database locally for security reasons since some classes has the same data as live.

I do believe we can still overcome this by having multiple Caché databases (one for each developer) on our server instance (instead of namespaces and web applications) and then map each local instance data to that of the server.

This would still require a connection to the server to have data in the application. This could work as mostly local but the data, no?

[isc-bsaviano]

If all you had were classes and routines then you could give each dev their own namespace on the shared server and then map the test data into each namespace. The web app is what complicates this. Does each dev really need this whole sensitive database to test their code? If not then they can have their own servers and development source control branches, and you can have one "test" server that has the full test data and a copy of the "main" source control branch that is ready to be promoted to "live". You will then validate changes at production scale after they have gone through code review and testing on a smaller scale.

[GabsReDeal]

Exactly, it is the webapp that complicates this, and why I wrote this mapping logic into the extension.

Unfortunately some of this data is required, as the application won't run properly without it. Like I said, it is a legacy system which started being developed around 26 years ago and a lot of bad crucial legacy code exists. I would have liked to identify all the data required and produce dummy data for it, but it is big task as most classes are written incorrectly where even relationships are just object properties instead.

I would like to push a proper source control system much sooner than that task would take to complete, especially since the team is getting larger quickly and everyone developing on one system (as we currently are) is a mess.

[isc-bsaviano]

If you must share a development server then you might want to use server-side source control (like Embedded Git) for now. This will allow you to edit files directly on the server like Studio and developers can share a namespace/web app. If you go with Embedded Git then it should be easy to switch to client-side editing (files on your local file system) when you can support private development environments. Client-side editing does not work well with shared development environments.

[GabsReDeal]

I have little to no experience with server-side source control, and the little I have I found tedious. And the only reason we would have the "shared" development server would be to hold a database for each developer which is not on the developers' workstation.

I want to move away from a shared development environment. Without this implementation, I believe that what I said earlier might still work for the time being as a hybrid local/server-side development. At least until we completely move to local, whenever that happens.

I do believe we can still overcome this by having multiple Caché databases (one for each developer) on our server instance (instead of namespaces and web applications) and then map each local instance data to that of the server.

Thanks

[isc-bsaviano]

I do believe we can still overcome this by having multiple Caché databases (one for each developer) on our server instance (instead of namespaces and web applications) and then map each local instance data to that of the server.

Just to clarify, you plan to have once instance of Cache for each developer on the same machine. That machine also has your "shared" Cache instance with the sensitive data. You then plan to map that sensitive database to each of the development servers. This will effective create "private" development environments, but with the shared tests data needed for the application. I believe that should work without any modification to this extension.

[GabsReDeal]

Thanks again for your prompt replies, I really appreciate it.

You are mostly correct. To clarify, each developer will have a Caché instance on their workstation (locally). This sensitive data would be in a shared Caché instance but duplicated for each developer on the server instance. Therefore, each developer would still have their own data, but it just is not locally on their machine.

This is similar to what I had in mind originally with the workspaces, but just with data being on the shared Caché instance. Unsure if this is possible, but the original multiple namespace version would've worked as a local/server-side kind of hybrid (with this pull request that is).

To reiterate, this is what I was thinking of when writing that message.

A development server which has a shared Caché instance that contains a database for each developer.
Each developer has a Caché instance on their workstation containing namespaces and webapps just like the Test and Prod instances.

[isc-bsaviano]

I think that should work as well. I am glad that we found a few paths that might work for your team. I am going to close this PR because I don't intend to merge it. The extensions in the official pack are supported by InterSystems, which means the barrier for entry is very high. We have an internal roadmap and goals that we need to measure all changes against. I updated our contributing guidelines to clarify expectations to save contributors time in the future. I do appreciate your effort and the interest you've taken in this project!

[GabsReDeal]

I will try to set it up this way and see how it goes. Thanks for your time.

[isc-bsaviano]

Happy to help! Please open another issue if you have any other questions.