Shiny module: automatic trigger from observeEvent

I'm developing a UI for a R program I created, and I have problem an observeEvent inside a module.

I have a UI module composed of a text input and a button.
On the module's server, I add an observeEvent bound to button from module's UI.
This observeEvent remove the text input and the UI when the button is clicked.

In the main app, I use a button to insert an instance of the module (insertUI).
When clicked, the UI element from the module are inserted, but the observedEvent is automatically triggered and elements are removed instantly.
Even by adding the ignoreInit = TRUE, I can't figure out how to avoid this automatic trigger.

Any ideas to avoid that?
Here a sample code to reproduce the behavior:

library(shiny)
library(shinymaterial)

##### Module #####
featureUI <- function(id) {
  ns <- NS(id)
  
  tagList(
    tags$input(
      id   = ns("feature"), 
      type = "text"), 
    material_button(
          input_id = ns("remove"), 
          label    = "RM", 
          icon     = "exposure_neg_1", 
          depth    = 2)
  )
}

featureServer <- function(id, num) {
  moduleServer(id, function(input, output, session) {
    
    observeEvent(input$remove, {
      cat("input$remove = ", input$remove, "\n")
      removeUI(selector = paste0("#iFeature", num, "-feature"))
      removeUI(selector = paste0("#iFeature", num, "-remove"))
    }, ignoreNULL = TRUE, ignoreInit = TRUE)
  })
}
##### Module #####



##### App Main #####
ui <- material_page(
  title = "Test",
  material_row(
    material_column(width = 2, offset = 0, 
      material_card(title   = "Update UI Test", depth   = 2, divider = TRUE, 
        material_number_box(input_id = "iWindow",     label = "Window length", min_value = 1000, max_value = 5000000, step_size = 100, initial_value = 50000), 
        material_button(    input_id = "iAddFeature", label = "Add", icon = "exposure_plus_1", depth = 2)
      )
    )
  )
)

server <- function(input, output, session) {

    observeEvent(input$iAddFeature, {
    fID <- paste0("iFeature", input$iAddFeature)
    insertUI(
      selector = "#iAddFeature", 
      where    = "beforeBegin", 
      ui       = featureUI(fID))
    
    featureServer(fID, input$iAddFeature)
  }, ignoreInit = TRUE)
}

shinyApp(ui = ui, server = server)

I don't think this will completely solve all your issues, but as far as shiny buttons go, they always get initialised to 0 value, so you could skip doing anything if the button value is zero.

 observeEvent(input$remove, {
      cat("input$remove = ", input$remove, "\n")
if(input$remove>0){
      removeUI(selector = paste0("#iFeature", num, "-feature"))
      removeUI(selector = paste0("#iFeature", num, "-remove"))
}
    }, ignoreNULL = TRUE, ignoreInit = TRUE)
  })

Thanks for this answer.
I thought to this solution, but i lead to weirder situation.
After this modification, the elements are added correctly, but the buttons becomes unresponsive.
I cannot understand why.

yes, I noticed that.
I thought it was strange that you dont assign the serverside initialisations of your modules to a name ?
I'm thinking perhaps they are being garbage collected away.
Don't you need to keep track of what modules you spin up so that you can harvest them for info ?

You mean storing the return of featureServer(fID, input$iAddFeature) in a reactiveValues()?
It's the first time I'm playing with modules, I might missed something.