Internet of Things (IoT) is becoming integral and mandatory part of everyday life. Scalability and manageability is intimidating due to unbounded number of devices and services. Access control and authorization in IoT with least privilege is important to establish secure communication between multiple devices and services. In this paper, the concept of capability for access control is introduced where the identities of the involved devices are entrenched in the access capabilities. Identity driven capability based access control (ICAC) scheme presented in this paper helps to alleviate issues related to the complexity and dynamics of device identities. ICAC is implemented for Wi-Fi and results shows that ICAC is scalable and performs better compared to other access control schemes. The ICAC evaluation by using security protocol verification tool shows that ICAC is secure against man-in-the-middle attack, especially eavesdropping and replay attacks.