An alliteration addict from Brazil has asked whether the notion of alliteration has to be purely sound-based. He had circulated a poem in which every line contained at least three alliterative words, and received a comment from one reader that some of his lines weren't really alliterative at all because they began with the same letter but not always with the same sound. For example, one of his lines reads 'Hope and humility are honorable'; another reads 'Always in artistic appreciations'.
In fact the earliest uses of the term in English refer only to letters, not sounds. The first citation in the OED is 1656, a definition which says that alliteration is 'a figure in Rhetorick, repeating and playing on the same letter'. One of the first quotations given in that dictionary is 'Apt Alliteration's artful aid' (1763), where the four instances of letter a represent three sounds.
The OED definition doesn't actually answer the question. It defines alliteration as: 'The commencing of two or more words in close connexion, with the same letter, or rather the same sound', Alliterate, however, has no qualification: 'To begin with the same letter or group of letters'.
Certainly, the way alliteration has been used in oral poetry (from Old English times) and in the oral performance of poetry has privileged the auditory sense of the term, and that is the dominant use today. Few people, I think, would view a sequence such as 'Peter the philosopher saw a ptarmigan' as alliterative.
What is the difference, then, between 'Always in artistic appreciations', which does sound alliterative, and 'Peter the philosopher...', which doesn't? The amount of phonetic similarity between the sounds. The phonetic values of the a letters are all relatively open vowels; the phonetic values of the p letters range from a plosive to a fricative to zero.
There is always an element of subjectivity in a judgement about alliteration (as indeed also about other effects, such as assonance and rhyme). Whether two sounds are perceived to alliterate depends on how close they are to each other, whether the two syllables are stressed or not, and - the issue here - whether the sounds are sufficiently phonetically similar to be perceived to be 'the same'. There are therefore no grounds for a 'black and white' interpretation of the notion. It would be unwise to insist that sounds in alliteration have to be always phonetically identical. A lot of effects that we recognize as alliterative (such as the OED example) would be excluded if we did.
There is another issue, of course: is it aesthetically acceptable to mix the two systems - graphic and phonic - in the same poem? That's perhaps more a question for literary criticism than linguistics. But if the aim of alliteration is not just to sound nice, but to relate word-meanings or to reinforce a poetic structure (such as parallelism between lines), then I don't see why not. Empson and others used to say that similarities of sound prompt similarities of sense. Similarities of shape can do the same thing.