I was thinking that it might be a one-player mallet part for Vibes and Marimba, so the Marimba part would orient the player. But even if it were a single line part, does a player really need something special to hold through four measures of 5/16? If so, then cues would be needed, no matter what the notation.That might work for pianists, but what about some notation that would be clear in single line orchestral parts as “a new kind of note …” as you suggested above?
But it is fun to contemplate an "whole-measure note." Such a note would only be necessary for meters in which it is not now possible to fill with a single note value, i.e. 5/x, 7/x, 9/x, 10/x, 11/x, 13/x, etc. In such cases, maybe some kind of a square whole note?