well, to sum it up, I'm talking about using regular old programming abstractions in place of language abstractions to generate an output program. This frees one from rigid language barriers.

Based on what you said, perhaps LLVM does similarly; but I'm guessing it's still about writing a grammar for which some source code must be specified (confined) within.

I argued that plain programming abstractions form a grammar of their own, but in an organic way (e.g. the very act of programming results in a custom grammar relevant to the task at hand, with words like "account" and "sendPayment"). This kind of "grammar" is not something separate from the code, so they take whatever form is meaningful as needed.

It's my assumption that a DSL does not quite provide the same utility, because you still have to specify a grammar separately from the code that code must adhere to. But perhaps if there is a more dynamic link between the two (e.g. your code can specify ad-hoc grammar rules directly amidst the code that uses them), then maybe that achieves the same thing I am after. Still though, I figure that if you have to say that XYZ(foo) translates into code ABC(bar,baz), then you may as well just write a function that takes arguments (foo) and returns the thing to generate (the AST node for ABC(bar,baz)).

I did start this thread in a different place than i ended up, and perhaps that needs to be reconciled:

There are lots of layers of building / generating / testing / etc. even on the "outside" of the actual code for the software product, and it all may as well be replaced with one program to do it all (because, arguably, that's what it all is). WiX (Windows Installer Xml) is a great example where a library would have been better than a language, especially since it's so incongruent to what actually gets generated by it. ... then I took this idea even further to say that even the source code for the actual product is just another layer within the grand specification of what to generate and what to do with the generated artifacts, so why not just one program to do it all?

Part of the utility comes in being able to specify parameters in one common code that affects how everything else (your the code, the databases, the service contracts, etc.) should jointly be affected (e.g. putting an $Environment token in each code, buy instead of running some build script to "replace" on everything, this token is a resolvable variable within one giant common program).

One other thing to reiterate is that I'm not so concerned with providing a grammar / DSL over TOP of an otherwise unchanging base layer (e.g. a program that gets translated to Java, still has the same limitations as Java), but moreso about replacing that hard-wired base (language) with a flexible layer of composable programming abstractions that you build up / customize as part of the source code (yours or via library). I still want to use the generating patterns and abstractions provided by languages, but in an ad-hoc manner. Sort of like COLA, though that does it via grammars (but as stated, maybe that's ok if it's dynamic/ad-hoc enough).

still confused about how this is not a thing you can do already. C and asm can call each other, and just about everything everywhere can call C (including haskell). use whatever language you think best fits a certain problem. if you think that language is limiting you somehow, use a different one, or split off just that subsection of the program to a different one. with a bit of effort, you can glue just about any language to any other language to produce programs that are written exactly how you want where you want. anybody can do this. people choose not to, when they can help it, however, because it's generally unnecessary complexity.

I revised my previous post, which may make it more clear.

But I'll repeat that this originated as "replace your build process with one big program", and then I roped in the idea that even the product source code itself is just another part of the larger build process, so why not include generation of the output artifact as part of the overall build-process-program so that it's all cohesive from to to bottom?

I took that further to say: Hey, this technically gets around any language barriers by letting you generate anything at all; and since programs contain abstractions, you can still use the same abstractions that you find in any language.

A POC will either demonstrate worth, or it will flop.

What you suggest about combining languages bears a surface resemblance to what I'm talking about, but it's very much not achieving nearly the same level of control/flexibility (i.e. control down to any ARBITRARY level). I am out of time now, but perhaps I'll paste a psuedocode later.

This topic was essentially just leading up to (or was one a many ideas in support of) creating an all-purpose dynamic software tool. Since it also ended my "Antelope" language (successor to Antidisassemblage), go view the post over on that thread to learn more:

https://www.cemetech.net/forum/viewtopic.php?p=264811#264811

Here is another explanation that I gave at Omnimaga, and the link between it and my new project:

Instead of writing a program in language-A or language-B and being limited to using only the features of one language, you should instead be able to mix and match whatever language features you want, even ones that you create yourself. Or you should be able to modify those features just for your program. So how do you do this?

Instead of using (or writing) a compiler for some language, you write your own program to generate the program you want. This would be more doable if you take the code that normally would be in a compiler, and instead make it part of a library. A compiler creates structural representations (AST) of the code that it compiles, so instead the library would be a tool to build parts of that representation and give them back to you. You could then modify them, or create your own somehow; and then you hand them back and say "ok, build the program from these".

Think of uncompiled source code (written in a compiled-language) as a program that makes an executable program, and the compiler as the interpreter of that program. When viewed in this way, declarations within code (classes, functions, variables) act as imperative commands to create components of a program. Replacing these "commands" with an API & user-defined functions would allow programs to be generated with any components & mechanisms, rather than from a fixed set of "language features". No more language restrictions!

::instead of this:: [tt]class Foo { Bar b }[/tt]
::you write this:: [tt]myClass = MakeClass("Foo"); myClass.AddMember("b", myBarClass)[/tt]

::another example:: [tt]myTrait = new MyVeryOwnTraitsImplementation("ResizableThing"); myTrait.ApplyTo(myClass)[/tt]

This is what makes languages like JavaScript so powerful: many things that are "declarations" in other languages are actually imperative calls to user-defined functions. This is why people have been able to make so many different kinds of object-models (etc.) in it. This makes things like compiler hooks and reflection mostly obsolete.

----

Now you might notice that the program that is generated by your program has whatever features and structure you want; but what about the program that did the generating; what language is that written in? Two possible answers are:
(1) It doesn't really matter, so long as you can generate whatever you want from it.
... However, since it just has to "run" once, it is more important for that code to have good expression than for it to be efficient, and that screams "interpreted language". Also, why would you want to have your source program, feed it into another program called a compiler, which generates an executable program, which you run to have it generate the program that you care about? Why not just run an interpreted program that generates the output program? So though it really does not matter what language the program-generating-program is written in, a compiled-language has no gain here, but the advantages of an interpreted language do apply.
(2) The only other option is to "close the loop" with some meta-circularness. That is, a software system or tool that is self-defining, self-modifying, etc., which is only possible if everyrthing (interpreter, code structure, etc.) is accessible at runtime. That is what my Objects project is about, except that it will do more than just that (e.g. a UI through which all things can be viewed or changed at runtime by the user, including the workings of the UI itself).

I see that I have a tendency to repeat myself rather than providing much new info. Though not my intent, it's probably frustrating and condescending. Sorry about that!

I can provide a bit more on why this is different from DSLs or runtime libraries or language interop, but I'll have to come back & do that later.

A (compiled) language assembles a program together using a set handful structural of patterns. Some of those structures can be designated by the programmer (e.g. static initialization blocks) and/or accessed & modified by the running program, and others cannot (e.g. vtables generated for each class). This despite the fact that they are all essentially the same stuff. Why should the programmer (i.e. the one who is "making" the thing) not have full ownership & ability over *all* of it? This is the bottleneck imposed by the programming language / compiler.

What I'm talking about is whether there can be some tool or technique that gets around this lockdown and lets you model and generate *all* structure. My claim is that regular programming abstractions are already well-suited for this.

DSLs, transpilers, LISP macros, C++ templates, and other source-code modification/generation can do *nothing* to alter the structural patterns in the output program. (If you don't care about that and just want better representation of *source code*, then these tools are just fine).

Language interop does not get around this either, because the different code for each language is generated into different boxes (programs), each with a similar bottleneck. You pick laws-of-the-universe-A or laws-B for each "box", but you cannot get at or change the structural properties of either.

What if you want to add things like coroutines, generators, mixins/traits (that are composable at runtime), and your language does not "have" them? What if you know that simple tweaking of a vtable would allow a powerful new ability, and it would be certainly valid in the generated program, but the language "doesn't do that"?

Now you'd never hear that about linked-lists or hash-maps or configuration-files, because there are sufficient means to create them yourself, and in fact that ability is standard skill & expectation. But were that not the case, would we still be saying "It's good enough" because you can still make things happen? We certainly don't accept that now though, because we know better.

Yes, you can make things happen without what I'm talking about, and stuff *does* happen without it; but that doesn't mean it's great. The current way in which we limit programming to textual representations in locked-down languages is *dreadful* compared to what it could be or could have been even decades ago. I could share a mountain of resources on that point, but that's another topic.

My point is that what I'm talking about is absolutely possible and would open many new doors to what it means to make software; and we'd accept nothing less if we had it.