This has come up twice in as many days… how do you remove certificates from the certificate store in PowerShell?

The certificate provider is ultimately a read-only view of your certificates. It does help you retrieve certificates, however, which is an important step in ultimately removing one from a store.
 
To remove one, you’ll need to use the .NET APIs:

[cert:\CurrentUser\TrustedPublisher]
PS:200 > dir

    Directory: Microsoft.PowerShell.Security\Certificate::CurrentUser\TrustedPublisher

Thumbprint                                Subject
----------                                -------
FD48FAA9281A657DBD089B5A008FAFE61D3B32FD  CN=PowerShell User
A25800BB7577F5854B3823B82228D94140D0244E  CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington...
564E01066387F26C912010D06BD78D3CF1E845AB  CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington...
 
[cert:\CurrentUser\TrustedPublisher]
PS:201 > $cert = @(dir)[0]

[cert:\CurrentUser\TrustedPublisher]
PS:202 > $store = New-Object System.Security.Cryptography.X509Certificates.X509Store "TrustedPublisher","CurrentUser"
Suggestion: An alias for New-Object is new

[cert:\CurrentUser\TrustedPublisher]
PS:203 > $store.Open("ReadWrite")

[cert:\CurrentUser\TrustedPublisher]
PS:204 > $store.Remove($cert)

[cert:\CurrentUser\TrustedPublisher]
PS:205 > $store.Close()

[cert:\CurrentUser\TrustedPublisher]
PS:206 > dir

    Directory: Microsoft.PowerShell.Security\Certificate::CurrentUser\TrustedPublisher

Thumbprint                                Subject
----------                                -------
A25800BB7577F5854B3823B82228D94140D0244E  CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington...
564E01066387F26C912010D06BD78D3CF1E845AB  CN=Microsoft Corporation, O=Microsoft Corporation, L=Redmond, S=Washington...