Button inside a rowReorder datatable not working

I am using a datatable whose rows can be reordered. The datatable contains a column with a button for each row to delete the row. When the row reorder functionality works, the button click does not do anything. I want the row reorder functionality to work and also the button to remove the row on click. Can someone help me out with this.

My html code is

<table id="draggableTable" class="table table-striped table-bordered dt-rowReorder" cellspacing="0" width="100%">   
    <thead>                     
        <tr>
            <th class="text-center">Order#</th>
            <th class="text-center min-tablet">ID</th>
            <th class="text-center">Item</th>
            <th class="text-center min-tablet">Remove</th>
        </tr>
    </thead>    
    <tbody>
         <tr>
            <td class="text-center">1</td>
            <td class="text-center min-tablet">35</td>
            <td>ID2123424</td>
            <td class="text-center min-tablet">
                <a id="removeItem-35" type="button" href="/remove-item" class="btn btn-sm btn-primary">Remove</a>
            </td>
         </tr>
         <tr>
            <td class="text-center">2</td>
            <td class="text-center min-tablet">44</td>
            <td>ID2123124</td>
            <td class="text-center min-tablet">
                <a id="removeItem-44" type="button" href="/remove-item" class="btn btn-sm btn-primary">Remove</a>
            </td>
         </tr>
         <tr>
            <td class="text-center">3</td>
            <td class="text-center min-tablet">46</td>
            <td>ID2123687</td>
            <td class="text-center min-tablet">
                <a id="removeItem-46" type="button" href="/remove-item" class="btn btn-sm btn-primary">Remove</a>
            </td>
         </tr>
         <tr>
            <td class="text-center">4</td>
            <td class="text-center min-tablet">50</td>
            <td>ID212645</td>
            <td class="text-center min-tablet">
                <a id="removeItem-50" type="button" href="/remove-item" class="btn btn-sm btn-primary">Remove</a>
            </td>
         </tr>
    </tbody>
</table>

I have the following script files included

<script src="https://cdn.datatables.net/1.10.13/js/jquery.dataTables.min.js"></script>
<script src="https://cdn.datatables.net/responsive/2.1.1/js/dataTables.responsive.min.js"></script>
<script src="https://cdn.datatables.net/rowreorder/1.2.0/js/dataTables.rowReorder.min.js"></script>

Additional scripts used are

$(document).ready(function() {
    $('.dt-rowReorder').DataTable({
        rowReorder: {
            selector: 'tr'
        },
        columnDefs:[
            {
                targets: 0, 
                visible: true
            }
        ]       
    });

    $(document).on( 'click', 'a[id^="removeItem-"]' ,function (event){
        event.preventDefault();
        //codes to remove the the item
    });
});

Please help me out with a resolution. Thanks