No. The program sets the payments: US$15,000 per person and the US$2 million corporate gift, plus US$15,000 and US$1 million for each joining dependent. Any clawback comes from your employment agreement, not from the official instructions, and only the US$15,000 processing fee is described as nonrefundable. Locate the clause before arguing about the amount. A repayment obligation lives in an employment agreement, an offer letter, a sponsorship side letter, or a separation agreement, and it either exists in one of those documents or it does not exist at all.
Where a reimbursement demand actually comes from
Read the clause your employer relies on and check what it defines as recoverable. For you, your spouse and your daughter the sponsored package totals US$4,045,000, paid at different stages and for different reasons, so a single lump-sum demand may not match the file. The 1% annual maintenance charge and the 5% transfer charge are corporate obligations; whether your contract shifts any to you is a question for counsel.
Check three things in whatever clause is produced: what event triggers repayment, how the recoverable sum is calculated, and whether the obligation applies where the company ends the employment rather than the employee. Clauses drafted for resignations are frequently invoked after a layoff, and the wording often does not stretch that far. Ask for the calculation in writing, itemised against the payments the company actually made.