Framework is a big word for what is going on here. But testing utility, or testing application seem no better. Anyway, for years my testing "framework" has been more or less:

RunTests←{
     f←'T'⍵.⎕NL ¯3
     b←⍵{
        b←⍺⍎⍵,' 0'
        ⎕←⍵,': ',b⊃'Passed' '*** Failed ***'
        b}¨f
     tf←+/b
     r←⊂'Tests run: ',⍕≢b
     r,←⊂'Passed: ',⍕+/~b
     r,←⊂'Failed: ',⍕+/b
     ⊃r
 } 

This runs on a namespace full of test functions that return a 1 or 0 for success and failure respectively. I would just copy this snippet to each new project, making a little application specific modification here and there. This could use a little improvement, but I don't want it to become a huge, unwieldy framework with lots of options, classes, methods, etc. I don't really want to write tests for my test framework, or much documentation. But I do want it to report code coverage, to have an option for stopping on failure, and to handle broken tests and other conditions.

My new and improved "framework" is now its own Github repository Provanto. Unfortunately, despite my best efforts, is has grown from one function of a dozen lines to 8 functions nearing a total of 100 lines:

:Namespace Provanto
 ⎕IO ⎕ML←0 1
 Assert←{
     0⊣'TEST FAILED'⎕SIGNAL 789/⍨~⍵
 }
 Coverage←{
     ~⍵.Profile:¯1 ''
     s←Spaces⍕⍵.CodeSpace
     n←s.⎕NL⊂-3 4
     p←'.',¨⍨⍕¨s
     af←⊃,/p{⍺∘,¨⍵}¨n
     d←⎕PROFILE'data'
     b←d[;1]∊⊂⍬
     ef←b/d[;0]
     el←(+\b)⊆d[;1]
     nr←(⎕NR¨af)~¨⊂⊂' }'
     al←⍳¨≢¨nr
     ul←al~¨(el,⊂⍬)[ef⍳af]
     c←100×1-÷/≢¨∊¨ul al
     k←0<≢¨ul
     uf←~af∊ef
     (uf/ul)←⊂⍬
     z←k/af{0=≢⍵:⍺ ⋄ ⍺,'[',(⍕⍵),']'}¨ul  ⍝
     c z
 }
 Display←{
     ⍵.Quiet>1:0
     r←⍵.Result
     h←'Number of tests:'(≢r)
     ⎕←'*'⍪(⍕h⍪⍵.Status,⍪+⌿r∘.=⍳5)⍪'*'
     ~⍵.Profile:0
     ⎕←'Code coverage: ',,'Q<%>I4'⎕FMT ⍵.Coverage
     ⍵.Coverage=100:0
     ⎕←'Untested code:'
     ⎕←↑⍵.Untested
     0
 }
 Exe←{
     r←⍺{
         0/⍨~⍺.Stop::1+789≠⎕DMX.EN
         ⍺.TestSpace⍎⍵,' 0'
     }⍵
     ⍺.Quiet>0:r
     r⊣⎕←(r⊃⍺.DecoratedStatus),' ',(⍕⍺.TestSpace),'.',⍵
 }
 Run←{
     ⍝ ⍺ ←→ [Stop 0|1 [Quiet 0|1|2]]
     ⍝ ⍵ ←→ Test space, [Code space]
     ⍝ ← ←→ Result space
     ⍺←0
     z←⎕NS''
     z.(Stop Quiet)←2↑⍺
     z.(TestSpace CodeSpace)←2↑⍵,0
     z.(Status DecoratedStatus)←Status''
     z.Profile←z.CodeSpace≠0
     _←z.TestSpace.⎕FX¨⎕NR¨'Assert' 'Try'
     z.Function←'T'z.TestSpace.⎕NL ¯3
     p←⎕PROFILE⍣z.Profile
     _←p¨'Clear'('Start' 'coverage')
     z.Result←z Exe¨z.Function
     _←p'stop'
     z.(Coverage Untested)←Coverage z
     _←p'clear'
     1:z←z⊣Display z
 }
 Spaces←{
     ⍵≢⍕⍎⍵:⍬
     s←⍵∘,¨'.',¨(⍎⍵).⎕NL ¯9
     0=≢s:,⍎⍵
     ∊(⍎⍵),∇¨s
 }
 Status←{
     s←'Passed' 'Failed' 'Broken' 'N/A' 'Disabled'
     d←↓(↑':',¨⍨s),' ',↑3/¨' !!--'
     s d
 }
 Try←{
     ⍺←⊢
     0::⎕DMX.EN
     0⊣⍺ ⍺⍺ ⍵
 }
:EndNamespace

The API is limited to the Run function:

      {Z}←[X [Y]] Provanto.Run A [B]  

Where A is a namespace full of test functions, B is an optional namespace full of code that triggers a code coverage feature, X is a flag for stopping on failing or broken tests and Y is a flag for suppressing session output. Z, the shy result, is a namespace full of test results, if needed for further processing or reporting.

The run function injects a function Assert and an operator Try into the test namespace. In a test, the Assert function is called to the left of a naked guard. I first saw this technique, and use of word assert in this context, from the late great Roger Hui. Like Roger's, the Provanto Assert function uses only a right argument and does not embed the match function inside Assert. Thus a line in a test function looks like:

      Assert 4=2+2:

This reads a little better than putting a value to the left, and in addition it leaves open the option of applying other functions besides match to produce a boolean without additional logic. A test function is thus a gauntlet of assertions terminated by a 0:

TestPlus←{
     Assert 4=2+2:
     Assert 2 3 4≡1 2 3+1:
     Assert 5 6≡2+3 4:
     Assert 5=2 3+Try 4 5 6:
     Assert 11=2+Try'A':
     0
 }

The Try operator attempts to execute a function, trapping and returning any error:

       Assert 5=2 3+Try 4 5 6:

If a code namespace is provided, code coverage is computed with ⎕PROFILE, and functions (with an explicit list of lines) that are not executed by the tests are displayed in the session.

What more is needed? No doubt the Profile function could be cleaned up a bit. A large application might have multiple test namespaces, so we might want to have a cover function to run them all, or enhance Run to take multiple test namespaces.

The next step is to review Lars Stampe Villadsen's presentation from the Dyalog 2023 conference and get this stuff running automatically on Github.