How to remove dynamic inputs from input list in R Shiny

In a script I am creating actionLinks dynamically. After I have used them I want to remove them from the list of inputs. My script is:

library(shiny)

ui <- fluidPage(
  actionButton("remove", "Remove UI"),
  uiOutput('buttonsToBeRemoved')
)

server <- function(input, output, session)
{
  output$buttonsToBeRemoved <- renderUI(
  {
    buttons <- c("ab","bc","cd","de","ef")
    x0 <- list()

    for (i in (1:length(buttons)))
    {
      x1 <- list(actionLink(inputId=buttons[i], label=buttons[i]))
      x0 <- c(x0,x1)
    }

    tagList(x0)
  })

  observeEvent(input$remove,
  {
    print(names(input))
    removeUI(selector = "#ef")
    print(names(input))
  })
}

shinyApp(ui, server)

I use removeUI to remove the button with id 'ef', but this only removes the button from the screen and not from the input list. 'print(names(input))' in observeEvent gives the same set of inputs before and after removeUI.

Can anybody tell how to remove inputs from the input list?